Three-dimensional volume data segmentation method and device

Abstract

The invention discloses a three-dimensional volume data segmentation method and device, and relates to the field of image processing. Through combination of a machine learning model and a traditional algorithm, advantages of deep learning and the traditional algorithm are taken into consideration, a complex deep learning model can be replaced by a simple machine learning model, the method is more portable and easy to deploy and use, a user does not need to label a large number of labels and carry out a large number of training, and the user experience is improved. And the segmentation line of any frame in the tomography three-dimensional volume data can be obtained only by editing one segmentation line on one image frame. Moreover, the user uses a semi-automatic segmentation method during manual segmentation, that is, the user only needs to click several coordinates on an image frame, and then rapid and accurate segmentation of the boundary of the target layer can be completed. The algorithm utilizes the supervision quantity manually input by the user, is equivalent to a weak supervision algorithm, and compared with a traditional algorithm, the algorithm is more accurate, has better generalization ability and can be suitable for tomography three-dimensional body data of various different sources and different conditions.

Description

Technical Field

[0001] The present application relates to the field of image processing, and in particular, to a method and apparatus for segmenting three-dimensional volume data. Background Art

[0002] Tomographic three-dimensional volume data may refer to OCT three-dimensional volume data obtained by Optical Coherence Tomography (OCT), OCTA three-dimensional volume data obtained by Optical Coherence Tomography Angiography (OCTA), or CT three-dimensional volume data obtained by computed tomography (abbreviated as X-CT or CT).

[0003] In order to provide a more accurate imaging basis, it is necessary to segment each frame of the tomographic three-dimensional volume data. The layer segmentation method based on deep learning requires a large number of accurate labels to obtain a suitable model, but the generalization ability of this model sometimes has certain deficiencies, that is, it will be affected by tomographic three-dimensional volume data from different sources and different situations. After all, it is difficult to consider all situations with a large number of accurate labels; while the generalization performance of the layer segmentation algorithm based on traditional algorithms is even worse, and it is more difficult to produce good segmentation results for all tomographic three-dimensional volume data. Summary of the Invention

[0004] The embodiments of the present application provide a method and apparatus for segmenting three-dimensional volume data, so as to achieve the purpose of improving the accuracy of the segmentation result.

[0005] To solve the above technical problems, the embodiments of the present application provide the following technical solutions:

[0006] Receive at least two anchor points input on the first image frame in the tomographic three-dimensional volume data, where the tomographic three-dimensional volume data includes multiple image frames;

[0007] Generate a first layer of segmentation lines based on the anchor points in the first image frame, and the first layer of segmentation lines includes the segmentation data of the first image frame;

[0008] Input the segmentation data of the first image frame into a machine learning model for training, and obtain the segmentation lines of the target image frames in the tomographic three-dimensional volume data based on the trained machine learning model.

[0009] Optionally, the segmentation data includes the labels and feature vectors of the points on the segmentation line, the labels and feature vectors of the preselected points in the upper layer of the segmentation line, and the labels and feature vectors of the preselected points in the lower layer of the segmentation line.

[0010] Optionally, obtaining the segmentation line of the target image frame in the tomographic three-dimensional volume data based on the trained machine learning model includes:

[0011] Determining the second-layer segmentation line of the second image frame based on the trained machine learning model, where the second-layer segmentation line includes the segmentation data of the second image frame;

[0012] Inputting the segmentation data of the second image frame into the trained machine learning model to perform iterative optimization training on the trained machine learning model;

[0013] Obtaining the segmentation line of the target image frame in the tomographic three-dimensional volume data based on the machine learning model after iterative optimization training.

[0014] Optionally, obtaining the segmentation line of the target image frame in the tomographic three-dimensional volume data based on the trained machine learning model includes:

[0015] Obtaining the known segmentation lines in the adjacent frames of the target image frame. For each point on the known segmentation line, using the corresponding coordinates of each point in the target image frame and multiple coordinates obtained by expanding up and down based on the corresponding coordinates as the points to be determined;

[0016] Obtaining the feature vectors of each point to be determined in the target image frame;

[0017] Inputting the feature vectors into the trained machine learning model to obtain the prediction results of the points to be determined corresponding to the feature vectors;

[0018] Determining at least two anchor points based on the prediction results of each point to be determined;

[0019] Generating the segmentation line of the target image frame based on the anchor points in the target image frame.

[0020] Optionally, the method for generating the segmentation line based on the anchor points includes:

[0021] Setting a path mask according to the anchor points;

[0022] Determining the starting point and the ending point in the path mask and calculating the weight map of the path mask;

[0023] Generating an adjacency matrix according to the path mask and the weight map;

[0024] Based on the starting point, the ending point and the adjacency matrix, obtaining the segmentation line using the shortest path search algorithm.

[0025] Optionally, setting the path mask according to the anchor points includes:

[0026] Taking the anchor points and multiple coordinate points obtained by expanding up and down based on the anchor points as path points;

[0027] Determine the feasible path range between adjacent path points according to the path direction, and combine all the feasible path ranges to obtain a path mask.

[0028] Optionally, after generating the segmentation line of the target image frame based on the anchor points in the target image frame, the method further includes:

[0029] Select the best segmentation point for each column in the target image frame according to the coordinates of each point to be determined, the confidence of the prediction result output by the machine learning model, and the known segmentation lines in the adjacent frames of the target image frame, and update the segmentation line of the target image frame.

[0030] Optionally, after generating the segmentation line of the target image frame based on the anchor points in the target image frame, the method further includes:

[0031] Generate a fitting curve according to the known segmentation lines in the adjacent frames of the target image frame;

[0032] Perform curve constraint correction on the segmentation line of the target image frame according to the fitting curve to obtain a curve constraint correction result;

[0033] Select the best segmentation point for each column in the target image frame according to the coordinates of each point to be determined, the confidence of the prediction result output by the machine learning model, the known segmentation lines in the adjacent frames of the target image frame, and the curve constraint correction result, and update the segmentation line of the target image frame.

[0034] Optionally, performing curve constraint correction on the segmentation line of the target image frame according to the fitting curve to obtain a curve constraint correction result includes:

[0035] Move the fitting curve up and down according to the segmentation line of the target image frame until the number of points with a coordinate distance less than the first threshold between the points on the segmentation line of the target image frame and the fitting curve is the largest and then stop moving;

[0036] For the image points whose distance between the points on the segmentation line of the target image frame and the moved fitting curve exceeds the second threshold, adjust the coordinates in the direction of the moved fitting curve to obtain a curve constraint correction result.

[0037] Optionally, obtaining the feature vectors of each point to be determined in the target image frame includes:

[0038] Perform one or more three-dimensional image processings such as three-dimensional Gaussian filtering, three-dimensional gradient calculation, three-dimensional maximum gradient calculation, three-dimensional mean filtering, and three-dimensional variance calculation on the tomographic three-dimensional volume data to obtain the three-dimensional image processing results of each point to be determined;

[0039] Construct feature vectors based on the three-dimensional image processing results of each point to be determined.

[0040] The present application also provides a three-dimensional volume data segmentation device, including:

[0041] A receiving module, configured to receive at least two anchor points input on a first image frame in tomographic scan three-dimensional volume data, where the tomographic scan three-dimensional volume data includes multiple image frames;

[0042] A generating module, configured to generate a first-layer segmentation line based on the anchor points in the first image frame, where the first-layer segmentation line includes segmentation data of the first image frame;

[0043] A machine learning model module, configured to input the segmentation data of the first image frame into a machine learning model for training, and obtain a segmentation line of a target image frame in the tomographic scan three-dimensional volume data based on the trained machine learning model.

[0044] As can be seen from the above technical solutions, the embodiments of the present application have the following advantages: By combining a machine learning model and an auxiliary OCT layer segmentation algorithm in a traditional algorithm, the present application takes into account the advantages of both deep learning and traditional algorithms, and can also replace a complex deep learning model with a simple machine learning model, which is more lightweight and easy to deploy and use. Users do not need to label a large number of tags and perform a large amount of training, but only need to edit a segmentation line on one image frame to obtain the segmentation line of any frame in the tomographic scan three-dimensional volume data. Moreover, when manually segmenting, users use a semi-automatic segmentation method, that is, users only need to click several coordinates on the image frame to quickly and accurately segment the boundary of the target layer. This algorithm utilizes the supervision amount manually input by users, which is equivalent to a weak supervision algorithm. Compared with traditional algorithms, it will be more accurate and have better generalization ability, and can be applied to tomographic scan three-dimensional volume data from a variety of different sources and different situations. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a flowchart of the first embodiment of the present application;

[0046] Figure 2 It is a flowchart of the second embodiment of the present application;

[0047] Figure 3 It is a path mask schematic diagram provided by the embodiment of the present application;

[0048] Figure 4 It is a segmentation result schematic diagram provided by the embodiment of the present application;

[0049] Figure 5 It is another segmentation result schematic diagram provided by the embodiment of the present application;

[0050] Figure 6 It is a flowchart of the third embodiment of the present application;

[0051] Figure 7A schematic diagram of an anchor point provided by an embodiment of the present application;

[0052] Figure 8 Another schematic diagram of a path mask provided by an embodiment of the present application;

[0053] Figure 9 A schematic diagram of a diffusion result provided by an embodiment of the present application;

[0054] Figure 10 A schematic diagram of a device of the present application;

[0055] Figure 11 A schematic diagram of a computer device of the present application. Detailed implementation manners

[0056] Next, the technical solutions in the embodiments of the present application will be clearly and completely described with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0057] Please refer to Figure 1 , the specific steps of the first embodiment of the present application are as follows:

[0058] S101: The computer receives at least two anchor points input on the first image frame in the tomographic scan three-dimensional volume data.

[0059] The tomographic scan three-dimensional volume data includes: OCT three-dimensional volume data, OCTA three-dimensional volume data, or CT three-dimensional volume data. The tomographic scan three-dimensional volume data includes multiple image frames.

[0060] The user can manually input at least two anchor points in any one image frame (i.e., the first image frame) in the tomographic scan three-dimensional volume data to perform layer segmentation at the position required by the user. Among them, the user can also generate multiple segmentation lines by inputting multiple anchor points to segment multiple layer boundaries in the image frame.

[0061] In some implementation manners, the user can click two boundary points (the boundary points specifically refer to the points located at the layer boundary or the points near the layer boundary) in the first image frame each time to complete the segmentation of one segment. By clicking on the first image frame multiple times, the user can complete the segmentation of the entire line, thereby performing layer segmentation on the first image frame. Therefore, the computer can receive two boundary points (i.e., anchor points) input by the user each time, perform segment segmentation on the first image frame to obtain segment segmentation lines, and connect the heads and tails of multiple segment segmentation lines to generate the first layer segmentation line.

[0062] In some other implementations, the user can also click on multiple points in the first image frame at once, thereby completing the segmentation of the entire line. Therefore, the computer can receive multiple anchor points input by the user, segment the first image frame, and generate a first-layer segmentation line.

[0063] S102: The computer generates a first-layer segmentation line based on the anchor points in the first image frame.

[0064] The first-layer segmentation line includes the segmentation data of the first image frame. Specifically, the segmentation data may include: the labels and feature vectors of the points on the segmentation line, the labels and feature vectors of the preselected points above the segmentation line, and the labels and feature vectors of the preselected points below the segmentation line.

[0065] After generating the first-layer segmentation line, the points on the first-layer segmentation line can be expanded up and down to obtain multiple sample points. Specifically, the points on the first-layer segmentation line can be expanded by multiple pixels up and down. Taking the coordinate value of a point on the first-layer segmentation line as (i, j, k) as an example, if three points above and below are expanded, the sample points may include: (i, j - 3, k), (i, j - 2, k), (i, j - 1, k), (i, j, k), (i, j + 1, k), (i, j + 2, k), and (i, j + 3, k). The number of sample points expanded up and down can be selected as needed, and the number of points expanded upward and the number of points expanded downward can be different.

[0066] Therefore, in the first image frame, the preselected points above the segmentation line refer to the sample points in the sample points of the first image frame that are located above the first-layer segmentation line; the preselected points below the segmentation line refer to the sample points in the sample points of the first image frame that are located below the first-layer segmentation line.

[0067] In some implementations, the label of each point can be set according to the position of each point. For example, the labels of the points on the segmentation line can be set to 1, and the labels of the preselected points above and below the segmentation line can be set to 0; the labels of the points on the segmentation line can be set to 0, the label of the preselected points above the segmentation line can be set to 1, and the label of the preselected points below the segmentation line can be set to 2; the labels of the points on the segmentation line and the labels of the preselected points above the segmentation line can be set to 0, and the label of the preselected points below the segmentation line can be set to 1; the label of the preselected points above the segmentation line can be set to 0, and the labels of the points on the segmentation line and the preselected points below the segmentation line can be set to 1. It should be noted that the setting method of the labels is not limited to the above examples and can be set according to actual needs. There are at least two types of labels, and any two of the segmentation line, the upper layer of the segmentation line, and the lower layer of the segmentation line can be distinguished.

[0068] In some other implementation manners, the computer can preprocess the three-dimensional tomographic data and construct the feature vectors of each point in the image frame based on the preprocessed three-dimensional tomographic data. Specifically, the computer can perform at least one of the following three-dimensional image processes on the three-dimensional tomographic data: three-dimensional Gaussian filtering, three-dimensional gradient calculation, three-dimensional maximum gradient calculation, three-dimensional mean filtering, and three-dimensional variance calculation. Therefore, the feature vectors of the points on the segmentation line, the preselected points in the upper layer of the segmentation line, and the preselected points in the lower layer of the segmentation line can be obtained through the following method: the computer performs at least one three-dimensional image process on the three-dimensional tomographic data to obtain the three-dimensional image process results of the points on the segmentation line, the preselected points in the upper layer of the segmentation line, and the preselected points in the lower layer of the segmentation line; based on the three-dimensional image process results of the points on the segmentation line, the preselected points in the upper layer of the segmentation line, and the preselected points in the lower layer of the segmentation line, construct the feature vectors of the points on the segmentation line, the preselected points in the upper layer of the segmentation line, and the preselected points in the lower layer of the segmentation line. It should be noted that those skilled in the art can also perform other preprocessing on the three-dimensional tomographic data according to actual requirements. The preprocessing is not limited to the above examples. The purpose of the preprocessing is to obtain the feature vectors. Therefore, the steps of the preprocessing can be operated as long as before the actual use of the corresponding feature vectors, and it is not limited to before or after the step of obtaining the first segmentation line.

[0069] In some other implementation manners, a path mask can be set according to the anchor points in the image frame, and the path range of the segmentation line can be defined by the path mask, so as to reduce path interference, make the generated segmentation line more accurate, and also reduce the calculation amount and improve the processing efficiency. Specifically, the method for generating a segmentation line based on anchor points can include: the computer sets a path mask according to the anchor points; determines the start point and the end point in the path mask, and calculates the weight map of the path mask; generates an adjacency matrix according to the path mask and the weight map; based on the start point, the end point, and the adjacency matrix, obtains the segmentation line by using the shortest path finding algorithm. Therefore, "generating the first segmentation line based on the anchor points in the first image frame" can be specifically implemented in the following manner: the computer sets a path mask according to the anchor points in the first image frame; determines the start point and the end point in the path mask, and calculates the weight map of the path mask; generates an adjacency matrix according to the path mask and the weight map; based on the start point, the end point, and the adjacency matrix, obtains the first segmentation line by using the shortest path finding algorithm.

[0070] S103: The computer inputs the segmentation data of the first image frame into a machine learning model for training, and obtains the segmentation line of the target image frame in the three-dimensional tomographic data based on the trained machine learning model.

[0071] The machine learning model can be a random forest model, can be a support vector machine model, and the machine learning model can also be set as other models according to actual requirements.

[0072] The target image frames can include all the image frames in the tomographic three-dimensional volume data except the first image frame, or can also be some of the image frames or a certain image frame in the tomographic three-dimensional volume data except the first image frame.

[0073] The computer inputs the segmentation data of the first image frame into a machine learning model for training. The trained machine learning model can thus automatically generate segmentation lines for all the image frames in the tomographic three-dimensional volume data except the first image frame.

[0074] In some implementation manners, the trained machine learning model can first generate a segmentation line for the second image frame, and based on the segmentation line of the second image frame, perform iterative optimization training on the trained machine learning model. The second image frame can also be one of the multiple target image frames. The principle of obtaining the segmentation line of the second image frame is the same as that of obtaining the segmentation line of the target image frame. However, after obtaining the segmentation line of the second image frame, using its segmentation data to perform iterative optimization training on the machine learning model can enable the machine learning model to more accurately segment other target image frames. Therefore, "obtaining the segmentation line of the target image frame in the tomographic three-dimensional volume data based on the trained machine learning model" includes: the computer determines the second-level segmentation line of the second image frame based on the trained machine learning model, and the second-level segmentation line includes the segmentation data of the second image frame; inputs the segmentation data of the second image frame into the trained machine learning model to perform iterative optimization training on the trained machine learning model; and obtains the segmentation line of the target image frame in the tomographic three-dimensional volume data based on the machine learning model after iterative optimization training. Wherein, the second image frame can be adjacent to the first image frame, or can also be an image frame separated from the first image frame by a preset number of frames.

[0075] In some other implementation manners, "obtaining the segmentation line of the target image frame in the tomographic three-dimensional volume data based on the trained machine learning model" is specifically implemented in the following manner: the computer obtains the known segmentation lines in the neighboring frames of the target image frame, and for each point on the known segmentation line, takes the corresponding coordinate of the point in the target image frame and multiple coordinates obtained by expanding up and down based on the corresponding coordinate as the points to be determined; obtains the feature vectors of each point to be determined in the target image frame; inputs the feature vectors into the trained machine learning model to obtain the prediction results of the points to be determined corresponding to the feature vectors; determines at least two anchor points based on the prediction results of each point to be determined; and generates the segmentation line of the target image frame based on the anchor points in the target image frame.

[0076] It should be noted that if the segmentation lines have been generated for multiple image frames in the tomographic scan three-dimensional volume data, the known segmentation line can be the segmentation line corresponding to the image frame that is the closest to the target image frame among the multiple image frames; and if the segmentation line is only generated for the first image frame in the tomographic scan three-dimensional volume data, the known segmentation line is the segmentation line corresponding to the first image frame.

[0077] In the first embodiment of the present application, by combining a machine learning model and an auxiliary OCT layer segmentation algorithm in a traditional algorithm, the advantages of both deep learning and traditional algorithms are taken into account. Moreover, a complex deep learning model can be replaced with a simple machine learning model, which is more lightweight and easier to deploy and use. Users do not need to label a large number of tags and conduct a large amount of training. Instead, they only need to edit a segmentation line on one image frame to obtain the segmentation lines of any frame in the tomographic scan three-dimensional volume data. And when users perform manual segmentation, they use a semi-automatic segmentation method, that is, users only need to click on a few coordinates on the image frame to quickly and accurately segment the boundary of the target layer. This algorithm utilizes the supervision amount manually input by users, which is equivalent to a weakly supervised algorithm. For traditional algorithms, it will be more accurate and have better generalization ability, and can be applied to tomographic scan three-dimensional volume data from a variety of different sources and different situations.

[0078] The generation of the segmentation line will be described below in combination with the specific setting method of the path mask. In the second embodiment, taking the first image frame as an example, it is explained how the first-layer segmentation line is generated. It should be noted that the method for generating the segmentation lines for the second image frame and the target image frame is also based on the same principle.

[0079] Please refer to Figure 2 , the specific steps of the second embodiment of the present application are as follows:

[0080] S201: The computer receives two anchor points input on the first image frame in the tomographic scan three-dimensional volume data.

[0081] In the second embodiment, only how to generate the path mask based on two anchor points is described. Among them, the input left point is g, and the input right point is r. Then the coordinates of the left point are (g.x, g.y), and the coordinates of the right point are (r.x, r.y).

[0082] The way of generating the path mask based on multiple anchor points is based on the same principle as the way of generating the path mask based on two anchor points. The path mask can also be generated in the way of this embodiment for two adjacent anchor points among multiple anchor points, which will not be elaborated below.

[0083] S202: The computer uses the anchor points in the first image frame and multiple coordinate points obtained by expanding up and down based on the anchor points in the first image frame as path points.

[0084] S203: The computer determines the feasible path range between adjacent path points according to the path direction, and combines all the feasible path ranges to obtain a path mask.

[0085] Through two anchor points, the intersection points p1 and p2 can be obtained. The coordinates of the intersection point p1 are (p1.x, p1.y), and the coordinates of the intersection point p2 are (p2.x, p2.y). Among them, p1 is the intersection point of the line passing through the left point g with a slope of 45 degrees and the line passing through the right point r with a slope of -45 degrees, and p2 is the intersection point of the line passing through the left point g with a slope of -45 degrees and the line passing through the right point r with a slope of 45 degrees. The slope can be set by itself. The intersection point is to obtain a feasible path from one anchor point to another anchor point. The area passed through above and below constitutes the feasible path range. The above intersection point can also be replaced by the boundary points of the upper and lower path ranges between two anchor points determined by other methods. For example, circles, ellipses, rhombuses, etc. are made with two anchor points, or circles and preset polygons are made with the points obtained by floating the two anchor points up and down.

[0086] In this embodiment, specifically, the two anchor points and the two intersection points can be respectively floated up and down by 3 pixels to obtain path points. By floating the above-mentioned anchor points and the intersection points obtained based on the anchor points up and down, the error correction of the anchor points manually input by the user can be performed, making the path mask more accurate. It should be noted that the floating pixel value can be set according to actual needs and is not limited to the 3 pixels exemplified above.

[0087] Please refer to Figure 3 , Figure 3 which is a schematic diagram of a path mask provided in this embodiment. Taking the number of anchor points in this embodiment as 2 as an example, the path mask may include the following six vertices: (g.x, g.y - 3), (p2.x, p2.y - 3), (r.x, r.y - 3), (r.x, r.y + 3), (p1.x, p1.y + 3), and (g.x, g.y + 3). All the coordinate points within the range of the polygon formed by the six vertices are used as path points.

[0088] The path direction can be from left to right. Please continue to refer to Figure 3 , Figure 3 The white part in the middle is the path mask, and the path mask includes: all the feasible path ranges between adjacent path points.

[0089] S204: The computer determines the start point and end point in the path mask and calculates the weight map of the path mask.

[0090] In this embodiment, based on the path mask obtained through the above steps, the starting points can be determined as: (g.x, g.y - 3), (g.x, g.y - 2), (g.x, g.y - 1), (g.x, g.y), (g.x, g.y + 1), (g.x, g.y + 2), (g.x, g.y + 3); and the ending points can be determined as: (r.x, r.y - 3), (r.x, r.y - 2), (r.x, r.y - 1), (r.x, r.y), (r.x, r.y + 1), (r.x, r.y + 2), (r.x, r.y + 3). It should be noted that any one of the above 6 starting points can reach any one of the above 6 ending points.

[0091] In some implementation manners, the computer can pre-judge whether the weight map corresponding to the segment division line obtained according to the left point g and the right point r is from bright to dark or from dark to bright. The judgment steps are as follows:

[0092] S2041: The computer calculates the grayscale mean difference between the upper and lower parts of the left point g within a preset range to obtain the first grayscale mean difference.

[0093] The preset range can be set to float 5 pixels up, down, left, and right. Then, the upper part of the left point g within the preset range is [g.x - 5, g.x + 5][g.y - 5, g.y], and the lower part of the left point g within the preset range is [g.x - 5, g.x + 5][g.y, g.y + 5].

[0094] The first grayscale mean difference = the grayscale mean of the upper part of the left point g within the preset range - the grayscale mean of the lower part of the left point g within the preset range.

[0095] It should be noted that the pixel value by which the preset range floats can also be set to other values according to actual needs, and is not limited to the 5 pixels exemplified above.

[0096] S2042: The computer calculates the grayscale mean difference between the upper and lower parts of the right point r within a preset range to obtain the second grayscale mean difference.

[0097] Still taking the pixel value by which the preset range floats as 5 pixels as an example, the upper part of the right point r within the preset range is [r.x - 5, r.x + 5][r.y - 5, r.y], and the lower part of the right point r within the preset range is [r.x - 5, r.x + 5][r.y, r.y + 5].

[0098] The second grayscale mean difference = the grayscale mean of the upper part of the right point r within the preset range - the grayscale mean of the lower part of the right point r within the preset range.

[0099] S2043: The computer adds the first grayscale mean difference and the second grayscale mean difference to obtain the third grayscale mean difference.

[0100] S2044: If the third grayscale mean difference is less than zero, the computer determines that the weight maps corresponding to the left point g and the right point r are from dark to bright.

[0101] S2045: If the third grayscale mean difference is greater than or equal to zero, the computer determines that the weight maps corresponding to the left point g and the right point r are from bright to dark.

[0102] In step S204, the computer can calculate the weight map according to the pre-judged result (the weight map is from dark to bright, or the weight map is from bright to dark).

[0103] In some other implementation manners, the computer can also perform the above judgment and calculation on the weight map in step S204.

[0104] S205: The computer generates an adjacency matrix based on the path mask and the weight map.

[0105] It should be noted that the path mask value of each point in the first image frame can be 0 or 1. If the path mask value is 0, it means that this point cannot be searched. Reflected in the adjacency matrix, the value corresponding to this point is set to a relatively large value such as infinity or 9999.

[0106] S206: The computer obtains the first-layer segmentation line by using the shortest path finding algorithm based on the starting point, the ending point, and the adjacency matrix.

[0107] It should be noted that the shortest path finding algorithm can search for points in the order from the upper right, the middle right to the lower right.

[0108] It should be noted that if the path mask value corresponding to the point (i, j) in the path mask is 0, no processing is performed on this point and it is directly skipped. At this time, it is not necessary to traverse each point in the first image frame, which speeds up the segmentation speed.

[0109] In some implementation manners, 6 paths can be obtained through the above shortest path finding algorithm, and the computer takes the path with the smallest path weight sum among the 6 paths as the first-layer segmentation line.

[0110] In some other implementation manners, after the computer obtains the first-layer segmentation line, it can also perform mean filtering processing on the first-layer segmentation line, so as to smooth the first-layer segmentation line. Specifically, the size of the mean filter kernel can be 5.

[0111] Please refer to Figure 4 , Figure 4 which is a schematic diagram of a segmentation result provided in this embodiment and is the segmentation result obtained by segmenting two anchor points based on the segmentation method of this embodiment. Please refer to Figure 5 , Figure 5Another schematic diagram of the segmentation result provided in this embodiment is the segmentation result obtained by segmenting multiple anchor points based on the segmentation method of this embodiment.

[0112] In the second embodiment of the present application, a path mask can be set based on the anchor points. Since the path mask is obtained by expanding the anchor points up and down, the error correction of the anchor points manually input by the user can be performed, and the obtained path mask has a high accuracy. At the same time, the path mask can include all the feasible path ranges between multiple path points and exclude large areas of irrelevant ranges in the image. Based on the path mask for the shortest path search, the interference of other points can be reduced, so the generation efficiency of the segmentation line can be improved.

[0113] The following describes a specific implementation manner for generating a segmentation line of the target image frame.

[0114] Please refer to Figure 6 , the specific steps of the third embodiment of the present application are as follows:

[0115] S601: The computer obtains the known segmentation lines in the adjacent frames of the target image frame. For each point on the known segmentation line, the corresponding coordinates of each point in the target image frame and multiple coordinates obtained by expanding up and down based on the corresponding coordinates are used as the points to be determined.

[0116] The multiple coordinates can be obtained by expanding 3 pixels up and down based on the corresponding coordinates. Taking the frame where the known segmentation line is located as index, for example, there is a point (i, j, index) on the known segmentation line, then the points to be determined can include: (i, j - 3, index), (i, j - 2, index), (i, j - 1, index), (i, j, index), (i, j + 1, index), (i, j + 2, index) and (i, j + 3, index). It should be noted that the pixel value for expanding up and down the corresponding coordinates can be set according to actual needs and is not limited to the 3 pixels in the above example.

[0117] S602: The computer obtains the feature vectors of each point to be determined in the target image frame.

[0118] In some implementation manners, "the computer obtains the feature vectors of each point to be determined in the target image frame" can be implemented in the following manner: The computer performs one or more three-dimensional image processings such as three-dimensional Gaussian filtering, three-dimensional gradient calculation, three-dimensional maximum gradient calculation, three-dimensional mean filtering, and three-dimensional variance calculation on the tomographic scan three-dimensional volume data to obtain the three-dimensional image processing results of each point to be determined; based on the three-dimensional image processing results of each point to be determined, a feature vector is constructed. By combining three-dimensional feature information, the feature vectors of each point to be determined are constructed, so that the layer boundary segmentation of the image frame can be more accurate.

[0119] Specifically, step S602 may include the following steps:

[0120] S6021: The computer performs three-dimensional Gaussian convolution processing on the three-dimensional computed tomography volume data to obtain a first processing result for each point in the three-dimensional computed tomography volume data.

[0121] The first processing result can be represented by gaussian3D. Therefore, the first processing result corresponding to any point in the three-dimensional computed tomography volume data can be represented by gaussian3D(i, j, k).

[0122] S6022: The computer performs three-dimensional gradient calculation on each point in the three-dimensional computed tomography volume data according to the first processing result to obtain a second processing result for each point in the three-dimensional computed tomography volume data.

[0123] The second processing result can be represented by grad3D. Therefore, the second processing result corresponding to any point in the three-dimensional computed tomography volume data can be represented by grad3D(i, j, k).

[0124] Specifically, the second processing result corresponding to any point in the three-dimensional computed tomography volume data can be calculated by the following formula:

[0125] gradx(i, j, k) = gaussian3D(i - 1, j, k) - gaussian3D(i + 1, j, k)

[0126] grady(i, j, k) = gaussian3D(i, j - 1, k) - gaussian3D(i, j + 1, k)

[0127] gradz(i, j, k) = gaussian3D(i, j, k - 1) - gaussian3D(i, j, k + 1)

[0128]

[0129] S6023: The computer performs three-dimensional maximum gradient calculation on each point in the three-dimensional computed tomography volume data according to the second processing result to obtain a third processing result for each point in the three-dimensional computed tomography volume data.

[0130] The third processing result can be represented by gradMax3D. Therefore, the third processing result corresponding to any point in the three-dimensional computed tomography volume data can be represented by gradMax3D(i, j, k).

[0131] Specifically, the maximum filter kernel range can be set to [7, 7, 5]. Therefore, the third processing result corresponding to any point in the tomographic three-dimensional volume data can be calculated by the following formula:

[0132]

[0133] Among them, the maximum filter kernel range can be set according to actual needs and is not limited to the above example.

[0134] S6024: The computer performs three-dimensional mean filtering on the tomographic three-dimensional volume data to obtain the fourth processing result of each point in the tomographic three-dimensional volume data.

[0135] Specifically, the mean filter kernel range can be set to [7, 7, 5]. By calculating the mean value within the filter kernel range, the fourth processing result mean3D can be obtained. At this time, the fourth processing result corresponding to any point in the tomographic three-dimensional volume data can be represented by mean3D(i, j, k).

[0136] Among them, the mean filter kernel range can be set according to actual needs and is not limited to the above example.

[0137] S6025: The computer performs three-dimensional variance calculation on the tomographic three-dimensional volume data to obtain the fifth processing result of each point in the tomographic three-dimensional volume data.

[0138] Specifically, the variance calculation range can be set to [7, 7, 5]. By calculating the variance within the set range, the fifth processing result std3D can be obtained. At this time, the fifth processing result corresponding to any point in the tomographic three-dimensional volume data can be represented by std3D(i, j, k).

[0139] Among them, the variance calculation range can be set according to actual needs and is not limited to the above example.

[0140] It should be noted that step S6024 and step S6025 can be executed simultaneously with steps S6021 to S6023, and there is no execution sequence restriction among them.

[0141] S6026: The computer constructs the feature vectors of each point to be determined in the target image frame based on the first processing result, the second processing result, the third processing result, the fourth processing result, and the fifth processing result of each point to be determined in the target image frame.

[0142] In some implementation manners, 11-dimensional feature vectors can be constructed for each point to be determined in the target image frame. Taking the point (i, j, k) as an example, the feature vector feature(i, j, k) can be constructed by the following formula:

[0143] f1 = mean3D(i, j - 2, k) - mean3D(i, j + 2, k)

[0144] f2 = mean3D(i, j - 4, k) - mean3D(i, j + 4, k)

[0145] f3 = mean3D(i, j - 8, k) - mean3D(i, j + 8, k)

[0146] f4 = mean3D(i, j - 16, k) - mean3D(i, j + 16, k)

[0147] f5 = std3D(i, j - 2, k) - std3D(i, j + 2, k)

[0148] f6 = std3D(i, j - 4, k) - std3D(i, j + 4, k)

[0149] f7 = std3D(i, j - 8, k) - std3D(i, j + 8, k)

[0150] f8 = std3D(i, j - 16, k) - std3D(i, j + 16, k)

[0151] f9 = mean3D(i, j, k)

[0152] f10 = std3D(i, j, k)

[0153] f11 = gradMax3D(i, j, k)

[0154] feature(i, j, k) = [f1, f2, f3, f4, f5, f6, f7, f8, f9, f10, f11]

[0155] It should be noted that the dimension of the feature vector can be set according to actual needs, and it is not limited to the 11 - dimension example above.

[0156] S603: The computer inputs the feature vector into the trained machine - learning model to obtain the prediction result of the point to be determined corresponding to the feature vector.

[0157] Taking the machine learning model as a random forest model as an example, since the random forest model used in this embodiment is a binary classification model, the prediction result output by the random forest model is a floating point number, and its range is 0 to 1. The prediction result represents both the label and the confidence level. If in the segmentation data of the first image frame, the label of the preselected point above the segmentation line is 0, and the labels of the points on the segmentation line and the preselected points below the segmentation line are 1, then when the prediction result is closer to 0, it means that the point to be determined is located above the segmentation line, and when the prediction result is closer to 1, it means that the point to be determined is on or below the segmentation line. The degree of closeness of the value to 0 or 1 represents the confidence level.

[0158] It should be noted that if the machine learning model is other models, the output prediction result may only include the predicted label, or may include the predicted label and the confidence level.

[0159] S604: The computer determines at least two anchor points based on the prediction results of each point to be determined.

[0160] In this embodiment, there are 7 points to be determined in each column of the target image frame. The computer can select the point with the maximum confidence level of the label belonging to the segmentation line in each column of the target image frame as the initial prediction result of the target image frame in this column according to the coordinates of each point to be determined and the confidence level of the prediction result output by the machine learning model, and use the point with the confidence level of the initial prediction result greater than 0.9 as the anchor point. In this embodiment, for simplicity of operation, without distinguishing between the label and the confidence level, directly take the maximum value of the prediction result as the initial prediction result of this column, and use the point with the prediction result greater than 0.9 as the anchor point.

[0161] In some implementation manners, the computer can save the initial prediction result (heights) and the confidence level (probabilityMaxs) corresponding to the initial prediction result.

[0162] Please refer to Table 1. Table 1 is a schematic diagram of a prediction result provided in this embodiment. Each cell in Table 1 represents the maximum value of the prediction result in each column of the target image frame. The target image frame in this embodiment has 512 columns, and only some examples are intercepted in the table. It should be noted that Table 1 is only an example.

[0163] Table 1

[0164] 0.5157 0.7033 0.7433 0.7223 0.8227 0.7027 0.895 0.915 0.864 0.8273 0.6007

[0165] S605: The computer generates a segmentation line of the target image frame based on the anchor points in the target image frame.

[0166] It should be noted that for a detailed description of step S605, reference can be made to the second embodiment. The two are based on the same principle and will not be elaborated here.

[0167] In some implementations, after step S605, this embodiment may further include step S606.

[0168] S606: The computer selects the optimal segmentation point for each column in the target image frame based on the coordinates of each point to be determined, the confidence of the prediction result output by the machine learning model, and the known segmentation lines in the neighboring frames of the target image frame, and updates the segmentation line of the target image frame.

[0169] This embodiment adds a condition for updating the segmentation line: If the maximum confidence of the prediction result output by the machine learning model for the first column or the last column of the target image frame is less than 0.7, then the segmentation line of the target image frame needs to be updated.

[0170] Specifically, step S606 may include the following steps:

[0171] S6061: The computer selects the optimal segmentation point for each column in the target image frame based on the coordinates of each point to be determined, the confidence of the prediction result output by the machine learning model, and the known segmentation lines in the neighboring frames of the target image frame, and updates the anchor points of the target image frame.

[0172] If the maximum confidence of the first column is less than 0.7, the computer updates the y coordinate of the anchor point of the first column to the j coordinate of the first column in the known segmentation line; if the maximum confidence of the last column is less than 0.7, the computer updates the y coordinate of the anchor point of the last column to the j coordinate of the last column in the known segmentation line.

[0173] Please refer to Figure 7 , Figure 7 which is a schematic diagram of an anchor point provided in this embodiment.

[0174] S6062: The computer sets a path mask based on the updated anchor points of the target image frame.

[0175] If the difference in the abscissa between two consecutive points g and r among the anchor points is less than or equal to a preset threshold, then 4 vertices are obtained: (g.x, g.y - 3), (r.x, r.y - 3), (r.x, r.y + 3), and (g.x, g.y + 3), and polygon filling is performed according to the order of the 4 vertices to obtain the path mask between point g and point r.

[0176] If the difference in the abscissa between two consecutive points g and r among the anchor points is greater than a preset threshold, then obtain 6 vertices in the manner of S202 and S203 in the second embodiment: (g.x, g.y - 3), (p2.x, p2.y - 3), (r.x, r.y - 3), (r.x, r.y + 3), (p1.x, p1.y + 3), and (g.x, g.y + 3), and perform polygon filling according to the order of the 6 vertices to obtain the path mask between point g and point r.

[0177] For the anchor points with a relatively small difference in abscissa between two points, only 4 vertices are used to set the path mask, which speeds up the generation speed of the path mask, thereby reducing the processing time.

[0178] Please refer to Figure 8 , Figure 8 which is another schematic diagram of the path mask provided in this embodiment. By generating the path masks between multiple anchor points, the global path mask of the target image frame can be obtained.

[0179] S6063: Determine the starting point and the ending point in the path mask, and calculate the weight map of the path mask.

[0180] S6064: Generate an adjacency matrix according to the path mask and the weight map.

[0181] S6065: Based on the starting point, the ending point, and the adjacency matrix, use the shortest path finding algorithm to obtain the segmentation line.

[0182] In some other implementation manners, after step S605, this embodiment may further include the following steps:

[0183] S607: Generate a fitting curve according to the known segmentation lines in the neighboring frames of the target image frame.

[0184] Since the layer boundaries corresponding to two adjacent image frames are relatively close, the curves of the segmentation lines of two adjacent image frames are also approximately close. At this time, the curve constraint correction of the segmentation line of the target image frame can be performed according to the known segmentation lines.

[0185] In some implementation manners, "generating a fitting curve according to the known segmentation lines in the neighboring frames of the target image frame" is implemented in the following manner: The computer performs n - order polynomial curve fitting on the known segmentation lines to obtain the fitting curve.

[0186] S608: Perform curve constraint correction on the segmentation line of the target image frame according to the fitting curve to obtain the curve constraint correction result.

[0187] Specifically, step S608 can be implemented through the following steps:

[0188] S6081: Move the fitting curve up and down according to the dividing line of the target image frame until the movement stops when the number of points with the coordinate distance between the points on the dividing line of the target image frame and the fitting curve being less than the first threshold is the largest.

[0189] It should be noted that the first threshold can be set according to actual needs.

[0190] S6082: For the image points whose distance between the points on the dividing line of the target image frame and the moved fitting curve exceeds the second threshold, the computer adjusts the coordinates in the direction of the moved fitting curve to obtain the curve constraint correction result.

[0191] It should be noted that the second threshold can be set according to actual needs.

[0192] Among the above image points, if there is an image point whose y coordinate is less than the y coordinate of the corresponding point on the fitting curve, the computer subtracts 1 from the y coordinate of the image point; if there is an image point whose y coordinate is greater than the y coordinate of the corresponding point on the fitting curve, the computer adds 1 to the y coordinate of the image point. By adjusting the coordinates of the image points, the curve constraint correction result (curveConstraintLine) can be obtained.

[0193] S609: The computer selects the best dividing point for each column in the target image frame according to the coordinates of each point to be determined, the confidence of the prediction result output by the machine learning model, the known dividing line in the adjacent frame of the target image frame, and the curve constraint correction result, and updates the dividing line of the target image frame.

[0194] In some implementations, for points in the initial prediction result with a low confidence level, such as points less than 0.4, the computer updates the y-coordinate of the point to the y-coordinate of the corresponding column in the curve constraint correction result or the known segmentation line. For points in the initial prediction result with a medium confidence level, such as points greater than or equal to 0.4 and less than or equal to 0.8, if the absolute value of the difference between the y-coordinate of the curve constraint correction result and the y-coordinate of the corresponding column in the known segmentation line is greater than 3, and the y-coordinate of the corresponding column in the curve constraint correction result is greater than the y-coordinate of the corresponding column in the known segmentation line, then the computer updates the y-coordinate of the point to the y-coordinate of the corresponding column in the known segmentation line + 1; if the absolute value of the difference between the y-coordinate of the curve constraint correction result and the y-coordinate of the corresponding column in the known segmentation line is greater than 3, and the y-coordinate of the corresponding column in the curve constraint correction result is less than or equal to the y-coordinate of the corresponding column in the known segmentation line, then the computer updates the y-coordinate of the point to the y-coordinate of the corresponding column in the known segmentation line - 1; if the absolute value of the difference between the y-coordinate of the curve constraint correction result and the y-coordinate of the corresponding column in the known segmentation line is less than or equal to 3, then the computer updates the y-coordinate of the point to the y-coordinate of the corresponding column in the curve constraint correction result. For points in the initial prediction result with a high confidence level, such as points greater than 0.8 and less than 0.9, if the absolute value of the difference between the y-coordinate of the curve constraint correction result and the y-coordinate of the corresponding column in the known segmentation line is less than 3, then the computer updates the y-coordinate to the y-coordinate of the corresponding column in the curve constraint correction result.

[0195] By updating the y-coordinates of the points in the above multiple cases, the computer realizes the update of the segmentation line of the target image frame.

[0196] In some other implementations, for points in the initial prediction result with a confidence level less than the first confidence threshold, the computer updates the point to the corresponding point in the curve constraint correction result.

[0197] In some other implementations, for points in the initial prediction result with a confidence level less than the first confidence threshold, if the absolute value of the difference between the y-coordinate of the point and the y-coordinate of the corresponding column in the known segmentation line is greater than 3, and the y-coordinate of the point is greater than the y-coordinate of the corresponding column in the known segmentation line, then the computer updates the y-coordinate of the point to the y-coordinate of the corresponding column in the known segmentation line + 1; if the absolute value of the difference between the y-coordinate of the point and the y-coordinate of the corresponding column in the known segmentation line is greater than 3, and the y-coordinate of the point is less than or equal to the y-coordinate of the corresponding column in the known segmentation line, then the computer updates the y-coordinate of the point to the y-coordinate of the corresponding column in the known segmentation line - 1.

[0198] In some other implementations, the computer also performs mean filtering on the updated segmentation line of the target image frame.

[0199] It should be noted that the computer can, after executing step S605, first execute step S606, and then execute steps S607 to S609.

[0200] Please refer to Figure 9, Figure 9 A schematic diagram of the diffusion result provided by this embodiment Figure 9 The 250th frame shown is the first image frame. The machine learning model can segment the 30th frame, 150th frame, 300th frame, and 380th frame (target image frames) based on the first image frame, thereby generating corresponding segmentation lines. In this embodiment, taking the first image frame as the 250th frame as an example, the machine learning model can first diffuse to the 249th frame and 251st frame, and diffuse frame by frame to both sides, thereby generating segmentation lines for multiple image frames in the tomographic three-dimensional volume data.

[0201] In some other implementation manners, the computer can generate new training data and training labels based on the diffusion result of the target image frame by the machine learning model, and thereby further train the machine learning model according to the newly generated training data and training labels. Among them, the second image frame is a training image frame selected from the target image frames, and the segmentation data and labels of the second image frame are used as the training data and training labels.

[0202] Specifically, the ways for the computer to generate new training data and training labels can include the following various situations:

[0203] 1. The computer generates training samples and training labels for all points on the segmentation line of the target image frame according to the construction method of the segmentation data of the first image frame.

[0204] 2. If there is a situation where the confidence of a certain column in the initial prediction result of the target image frame is relatively low, such as less than the second confidence threshold, for example, 0.2, etc., the computer generates training samples and training labels for the corresponding points of this column on the segmentation line of the target image frame according to the construction method of the segmentation data of the first image frame. This strategy can specifically enrich the training samples and reduce the probability of the subsequent model outputting results with low confidence.

[0205] 3. If there is a situation where the confidence of a certain column in the initial prediction result of the target image frame is within the confidence threshold range, the computer generates training samples and training labels for the corresponding points of this column on the segmentation line of the target image frame according to the construction method of the segmentation data of the first image frame. Among them, the confidence threshold range includes the confidence intermediate value, such as a confidence of 0.6, etc. This strategy can make the confidence of the output increase when similar samples with medium confidence appear later, and increase the proportion of high-confidence results.

[0206] It should be noted that other settings can also be made for the generation methods of the training data and training labels according to actual needs, and it is not limited to the above examples.

[0207] Specifically, the ways for the computer to train the machine learning model can include the following various situations:

[0208] 1. After the computer generates the segmentation line for each target image frame, it can use the target image frame as the second image frame to train the machine learning model once.

[0209] 2. After the computer generates the segmentation line for every preset number of target image frames, it can use the target image frame as the second image frame to train the machine learning model once. At this time, there is at least a preset number of frames between the second image frames.

[0210] 3. When the computer determines that the newly generated training data and training labels reach the sample threshold, it can use the target image frame corresponding to the newly generated training data and training labels as the second image frame to train the machine learning model once. At the same time, set the new sample count to 0 and start counting again.

[0211] 4. For the first N frames among multiple target image frames, the computer can use every n1-th frame as the second image frame to train the machine learning model once; for the last M frames of the target image frames, the computer can use every m1-th frame as the second image frame to train the machine learning model once.

[0212] 5. When the proportion of low confidence corresponding to a certain target image frame reaches the proportion threshold, the computer can train the machine learning model once. The specific values corresponding to the low confidence and the proportion threshold can be set according to actual needs.

[0213] 6. The computer can train the machine learning model a preset number of times and train again when any of the above 5 situations is triggered.

[0214] It should be noted that other settings can also be made for the training method of the machine learning model according to actual needs, not limited to the examples given above.

[0215] In some other implementation manners, the machine learning model can also stop generating the segmentation line for the undiffused image frames in the tomographic scan three-dimensional volume data according to the stopping mechanism.

[0216] Specifically, the stopping mechanism can include the following multiple situations: 1. If the number of anchor points obtained for a certain image frame is less than the anchor point threshold, stop the diffusion in this direction. 2. If the mean confidence or the maximum confidence of the initial prediction result of a certain image frame is less than the third confidence threshold, stop the diffusion in this direction. It should be noted that other settings can also be made for the stopping mechanism according to actual needs, not limited to the examples given above.

[0217] In the third embodiment of the present application, the known dividing lines in the adjacent frames of the target image frame can be used to correct the curve error of the dividing line of the target image frame, thereby improving the accuracy of the dividing line of the target image frame. If the maximum confidence of a certain column in the target image frame is small, it means that the prediction result of this column is less accurate at this time. By using the j coordinate of the known dividing line in this column as the y coordinate of the anchor point of this column, the dividing line of the target image frame can be corrected in this column; since the layer boundary between two adjacent frames is relatively close, if there is a point in the dividing line of the target image frame and the distance between this point and the fitting curve of the known dividing line is large, it means that there may be an error at this point, so this point can also be corrected.

[0218] Please refer to Figure 10 , this application provides a segmentation device 1000 for three-dimensional volume data, including: a receiving module 1001, a generating module 1002, and a machine learning model module 1003.

[0219] The receiving module 1001: is used to receive at least two anchor points input on the first image frame in the tomographic three-dimensional volume data, and the tomographic three-dimensional volume data includes multiple image frames.

[0220] The generating module 1002: is used to generate a first-layer dividing line based on the anchor points in the first image frame, and the first-layer dividing line includes the segmentation data of the first image frame.

[0221] The machine learning model module 1003: is used to input the segmentation data of the first image frame into the machine learning model for training, and obtain the dividing line of the target image frame in the tomographic three-dimensional volume data based on the trained machine learning model.

[0222] Optionally, the machine learning model module 1003 includes: a determining unit, a training unit, and an obtaining unit.

[0223] The determining unit: is used to determine the second-layer dividing line of the second image frame based on the trained machine learning model, and the second-layer dividing line includes the segmentation data of the second image frame.

[0224] The training unit: is used to input the segmentation data of the second image frame into the trained machine learning model to perform iterative optimization training on the trained machine learning model.

[0225] The obtaining unit: is used to obtain the dividing line of the target image frame in the tomographic three-dimensional volume data based on the machine learning model after iterative optimization training.

[0226] Optionally, the machine learning model module 1003 includes: an expanding unit, an obtaining unit, a determining unit, and a generating unit.

[0227] Expansion unit: used to obtain known dividing lines in adjacent frames of the target image frame. For each point on the known dividing line, the corresponding coordinates of each point in the target image frame and multiple coordinates obtained by expanding up and down based on the corresponding coordinates are used as points to be determined.

[0228] Acquisition unit: used to acquire the feature vectors of each point to be determined in the target image frame.

[0229] Optionally, the acquisition unit: specifically used to perform one or more three-dimensional image processings such as three-dimensional Gaussian filtering, three-dimensional gradient calculation, three-dimensional maximum gradient calculation, three-dimensional mean filtering, and three-dimensional variance calculation on the tomographic three-dimensional volume data to obtain the three-dimensional image processing results of each point to be determined; construct feature vectors based on the three-dimensional image processing results of each point to be determined.

[0230] The acquisition unit: is also used to input the feature vectors into the trained machine learning model and obtain the prediction results of the points to be determined corresponding to the feature vectors.

[0231] Determination unit: used to determine at least two anchor points based on the prediction results of each point to be determined.

[0232] Generation unit: used to generate the dividing line of the target image frame based on the anchor points in the target image frame.

[0233] Optionally, the generation module 1002 includes: a setting unit, a calculation unit, an adjacency matrix unit, and a path finding unit.

[0234] Setting unit: used to set a path mask according to the anchor points.

[0235] Calculation unit: used to determine the starting point and the ending point in the path mask and calculate the weight map of the path mask.

[0236] Adjacency matrix unit: used to generate an adjacency matrix according to the path mask and the weight map.

[0237] Path finding unit: used to obtain the dividing line by using the shortest path finding algorithm based on the starting point, the ending point, and the adjacency matrix.

[0238] Optionally, the setting unit: specifically used to use the anchor points and multiple coordinate points obtained by expanding up and down based on the anchor points as path points; determine the feasible path range between adjacent path points according to the path direction, and combine all the feasible path ranges to obtain the path mask.

[0239] Optionally, a three-dimensional volume data segmentation device 1000 further includes: an update module 1004.

[0240] Update module 1004: It is used to select the optimal segmentation point for each column in the target image frame based on the coordinates of each point to be determined, the confidence of the prediction result output by the machine learning model, and the known segmentation lines in the adjacent frames of the target image frame, and update the segmentation line of the target image frame.

[0241] Optionally, a three-dimensional volume data segmentation device 1000 further includes: an update module 1004, a fitting module 1005, and a correction module 1006.

[0242] Fitting module 1005: It is used to generate a fitting curve based on the known segmentation lines in the adjacent frames of the target image frame.

[0243] Correction module 1006: It is used to perform curve constraint correction on the segmentation line of the target image frame according to the fitting curve to obtain a curve constraint correction result.

[0244] Update module 1004: It is used to select the optimal segmentation point for each column in the target image frame based on the coordinates of each point to be determined, the confidence of the prediction result output by the machine learning model, the known segmentation lines in the adjacent frames of the target image frame, and the curve constraint correction result, and update the segmentation line of the target image frame.

[0245] Optionally, the correction module 1006: Specifically, it is used to move the fitting curve up and down according to the segmentation line of the target image frame until the number of points with a coordinate distance less than the first threshold between the points on the segmentation line of the target image frame and the fitting curve is the largest and then stop moving; for the image points whose distance between the points on the segmentation line of the target image frame and the moved fitting curve exceeds the second threshold, adjust the coordinates in the direction of the moved fitting curve to obtain a curve constraint correction result.

[0246] Regarding the device in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0247] It should be noted that: when the three-dimensional volume data segmentation device provided in the above embodiments realizes the three-dimensional volume data segmentation function, only the above-mentioned division of each functional module is used for illustration. In practical applications, the above functions can be allocated to different functional modules according to needs, that is, the internal structure of the three-dimensional volume data segmentation device is divided into different functional modules to complete all or part of the functions described above. In addition, the three-dimensional volume data segmentation device provided in the above embodiments and the three-dimensional volume data segmentation method embodiments belong to the same concept, and the specific implementation process can be seen in the method embodiments, which will not be elaborated here.

[0248] Please refer to Figure 11, this application also provides a computer device 1100, including: a processor 1101 and a memory 1102.

[0249] The processor 1101 is coupled to the memory 1102, and at least one computer program instruction is stored in the memory 1102. The at least one computer program instruction is loaded and executed by the processor 1101 so that the computer device implements the method for segmenting three-dimensional volume data.

[0250] Finally, it should also be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device.

Claims

1. A method for segmenting three-dimensional volume data, characterized in that, Including: Receiving at least two anchor points on a first image frame in tomographic three-dimensional volume data, where the tomographic three-dimensional volume data includes multiple image frames; Generating a first-layer segmentation line based on the anchor points in the first image frame, where the first-layer segmentation line includes segmentation data of the first image frame; Inputting the segmentation data of the first image frame into a machine learning model for training, and obtaining a segmentation line of a target image frame in the tomographic three-dimensional volume data based on the trained machine learning model.

2. The method according to claim 1, wherein The segmentation data includes labels and feature vectors of each point on the segmentation line, labels and feature vectors of preselected points in the upper layer of the segmentation line, and labels and feature vectors of preselected points in the lower layer of the segmentation line.

3. The method according to claim 1, wherein The obtaining the segmentation line of the target image frame in the tomographic three-dimensional volume data based on the trained machine learning model includes: Determining a second-layer segmentation line of a second image frame based on the trained machine learning model, where the second-layer segmentation line includes segmentation data of the second image frame; Inputting the segmentation data of the second image frame into the trained machine learning model to perform iterative optimization training on the trained machine learning model; Obtaining the segmentation line of the target image frame in the tomographic three-dimensional volume data based on the machine learning model after iterative optimization training.

4. The method according to claim 1, characterized in that The obtaining the segmentation line of the target image frame in the tomographic three-dimensional volume data based on the trained machine learning model includes: Obtaining known segmentation lines in adjacent frames of the target image frame, and for each point on the known segmentation line, taking the corresponding coordinates of the point in the target image frame and multiple coordinates obtained by expanding up and down based on the corresponding coordinates as points to be determined; Obtaining feature vectors of each point to be determined in the target image frame; Inputting the feature vectors into the trained machine learning model to obtain prediction results of the points to be determined corresponding to the feature vectors; Determining at least two anchor points based on the prediction results of each point to be determined; Generating a segmentation line of the target image frame based on the anchor points in the target image frame.

5. The method according to any one of claims 1 to 4, characterized in that The method for generating a segmentation line based on anchor points includes: Setting a path mask according to the anchor points; Determining a starting point and an ending point in the path mask, and calculating a weight map of the path mask; Generating an adjacency matrix according to the path mask and the weight map; Based on the starting point, the ending point, and the adjacency matrix, obtaining a segmentation line by using a shortest path finding algorithm.

6. The method according to claim 5, characterized in that, Setting a path mask according to the anchor points includes: Taking the anchor points and multiple coordinate points obtained by expanding up and down based on the anchor points as path points; Determining a feasible path range between each adjacent path point according to the path direction, and combining all feasible path ranges to obtain the path mask.

7. The method according to claim 4, wherein After generating the segmentation line of the target image frame based on the anchor points in the target image frame, the method further includes: Selecting the best segmentation point of each column in the target image frame according to the coordinates of each point to be determined, the confidence of the prediction result output by the machine learning model, and the known segmentation lines in adjacent frames of the target image frame, and updating the segmentation line of the target image frame.

8. The method according to claim 4, wherein After generating the segmentation line of the target image frame based on the anchor points in the target image frame, the method further includes: Generating a fitting curve according to the known segmentation lines in the adjacent frames of the target image frame; Performing curve constraint correction on the segmentation line of the target image frame according to the fitting curve to obtain a curve constraint correction result; Selecting the optimal segmentation point for each column in the target image frame according to the coordinates of each point to be determined, the confidence of the prediction result output by the machine learning model, the known segmentation lines in the adjacent frames of the target image frame, and the curve constraint correction result, and updating the segmentation line of the target image frame.

9. The method according to claim 8, wherein The performing curve constraint correction on the segmentation line of the target image frame according to the fitting curve to obtain a curve constraint correction result includes: Moving the fitting curve up and down according to the segmentation line of the target image frame until the number of points with the coordinate distance between the points on the segmentation line of the target image frame and the fitting curve less than the first threshold is the largest and then stopping the movement; For the image points whose distance between the points on the segmentation line of the target image frame and the moved fitting curve exceeds the second threshold, adjusting the coordinates in the direction of the moved fitting curve to obtain the curve constraint correction result.

10. The method according to claim 4, wherein The obtaining the feature vectors of each point to be determined in the target image frame includes: Performing one or more three-dimensional image processings such as three-dimensional Gaussian filtering, three-dimensional gradient calculation, three-dimensional maximum gradient calculation, three-dimensional mean filtering, and three-dimensional variance calculation on the tomographic three-dimensional volume data to obtain the three-dimensional image processing results of each point to be determined; Constructing the feature vectors based on the three-dimensional image processing results of each point to be determined.

11. A three-dimensional volume data segmentation device, characterized in that, Including: A receiving module, configured to receive at least two anchor points input on a first image frame in the tomographic three-dimensional volume data, where the tomographic three-dimensional volume data includes a plurality of image frames; A generating module, configured to generate a first-layer segmentation line based on the anchor points in the first image frame, where the first-layer segmentation line includes the segmentation data of the first image frame; A machine learning model module, configured to input the segmentation data of the first image frame into a machine learning model for training, and obtain the segmentation line of the target image frame in the tomographic three-dimensional volume data based on the trained machine learning model.

Images