Lung segment image determination method and device, computer equipment and storage medium

Abstract

The invention relates to a lung segment image determination method and device, computer equipment and a storage medium. The method comprises the following steps: segmenting a lung scanning image of a target object to obtain each bronchial segment segmentation image and each lung lobe segmentation image; inputting the lung scanning image into a vascular structure root node extraction model to obtain each vascular structure root node; inputting each bronchial segment segmentation image and each vascular structure root node into a lung lobe image section reconstruction model to obtain a lung lobe image section corresponding to each lung lobe segmentation image; and segmenting the corresponding lung lobe segmentation image according to the section of each lung lobe image to obtain each lung segment segmentation image. By adopting the method, the accuracy of lung segment segmentation in the lung image can be effectively improved.

Description

Technical Field

[0001] The present application relates to the field of medical imaging technology, and in particular to a method, apparatus, computer equipment, storage medium and computer program product for determining a lung segment image. Background Art

[0002] With the development of medical imaging technology, image observation and image extraction of the target object's lung image region of interest are both important. However, if you need to obtain the image of the region of interest, you need to segment the lung image. In traditional technology, lung segmentation of lung images is mainly divided into watershed method, deep learning method and machine learning method. However, traditional segmentation methods do not consider the complex tissue structure in the lung image during the lung segmentation process, resulting in insufficient accuracy in lung segmentation in lung images. Summary of the invention

[0003] Based on this, it is necessary to provide a lung segment image determination method, device, computer equipment, computer readable storage medium and computer program product that can effectively improve the accuracy of lung segmentation in lung images in order to address the above technical problems.

[0004] In the first aspect, the present application provides a method for determining a lung segment image. The method comprises: segmenting a lung scan image of a target object to obtain each bronchial segment segmentation image and each lung lobe segmentation image; inputting the lung scan image into a vascular structure root node extraction model to obtain each vascular structure root node; inputting each bronchial segmentation image and each vascular structure root node into a lung lobe image section reconstruction model to obtain a lung lobe image section corresponding to each lung lobe segmentation image; segmenting the corresponding lung lobe segmentation image according to each lung lobe image section to obtain each lung segmentation image.

[0005] In the second aspect, the present application also provides a lung segment image determination device. The device includes: a lung image segmentation module, which is used to segment the lung scan image of the target object to obtain each bronchial segment segmentation image and each lung lobe segmentation image; a vascular structure root node acquisition module, which is used to input the lung scan image into the vascular structure root node extraction model to obtain each vascular structure root node; a lung lobe image section acquisition module, which is used to input each of the bronchial segment segmentation images and each of the vascular structure root nodes into the lung lobe image section reconstruction model to obtain the lung lobe image section corresponding to each of the lung lobe segmentation images; a lung segment segmentation image acquisition module, which is used to segment the corresponding lung lobe segmentation image according to each of the lung lobe image sections to obtain each lung segmentation image.

[0006] In a third aspect, the present application also provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program: segmenting the lung scan image of the target object to obtain each bronchial segment segmentation image and each lung lobe segmentation image; inputting the lung scan image into the vascular structure root node extraction model to obtain each vascular structure root node; inputting each bronchial segmentation image and each vascular structure root node into the lung lobe image section reconstruction model to obtain the lung lobe image section corresponding to each lung lobe segmentation image; segmenting the corresponding lung lobe segmentation image according to each lung lobe image section to obtain each lung segmentation image.

[0007] In a fourth aspect, the present application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented: segmenting the lung scan image of the target object to obtain each bronchial segment segmentation image and each lung lobe segmentation image; inputting the lung scan image into a vascular structure root node extraction model to obtain each vascular structure root node; inputting each bronchial segment segmentation image and each vascular structure root node into a lung lobe image section reconstruction model to obtain a lung lobe image section corresponding to each lung lobe segmentation image; segmenting the corresponding lung lobe segmentation image according to each lung lobe image section to obtain each lung segmentation image.

[0008] In a fifth aspect, the present application also provides a computer program product. The computer program product includes a computer program, which implements the following steps when executed by a processor: segmenting the lung scan image of the target object to obtain each bronchial segment segmentation image and each lung lobe segmentation image; inputting the lung scan image into a vascular structure root node extraction model to obtain each vascular structure root node; inputting each bronchial segment segmentation image and each vascular structure root node into a lung lobe image section reconstruction model to obtain a lung lobe image section corresponding to each lung lobe segmentation image; segmenting the corresponding lung lobe segmentation image according to each lung lobe image section to obtain each lung segmentation image.

[0009] The above-mentioned lung segment image determination method, device, computer equipment, storage medium and computer program product obtain each bronchial segment segmentation image and each lung lobe segmentation image by segmenting the lung scan image of the target object; inputting the lung scan image into the vascular structure root node extraction model to obtain each vascular structure root node; inputting each bronchial segmentation image and each vascular structure root node into the lung lobe image section reconstruction model to obtain the lung lobe image section corresponding to each lung lobe segmentation image; segmenting the corresponding lung lobe segmentation image according to each lung lobe image section to obtain each lung segmentation image.

[0010] By introducing the location information of pulmonary arteries and veins, the lung scan image is segmented and segmented to obtain each bronchial segmentation image and each lung lobe segmentation image. Then the lung scan image is topologically analyzed to extract bifurcation points at all levels for the lung lobe image section. Finally, the lung lobe image section is used to segment the corresponding lung lobe segmentation image to obtain each lung segmentation image, so that the distance from the lung lobe image section to the adjacent bronchial segmentation structure or accompanying artery is basically equal, thereby ensuring that the section passes through the intersegmental vein to the greatest extent, effectively improving the accuracy of lung segment segmentation in the lung image. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 A diagram of an application environment of a lung segment image determination method in one embodiment;

[0012] Figure 2 A schematic diagram of a flow chart of a lung segment image determination method in one embodiment;

[0013] Figure 3 A schematic diagram of a process of a lung lobe image section reconstruction method in one embodiment;

[0014] Figure 4 It is a flowchart of a method for obtaining a bronchial segment segmentation image in one embodiment;

[0015] Figure 5 It is a flowchart of a method for obtaining lung lobe image section information in one embodiment;

[0016] Figure 6 It is a flowchart of a method for obtaining a point set of boundary lines for pulmonary lobe reconstruction in one embodiment;

[0017] Figure 7 A schematic flow chart of a method for obtaining a point set of boundary lines for pulmonary lobe reconstruction in another embodiment;

[0018] Figure 8 A schematic diagram of a flow chart of a method for obtaining lung lobe image section information in another embodiment;

[0019] Fig. 9 A schematic diagram of a flow chart of a lung lobe image section reconstruction method in another embodiment;

[0020] Fig.10 A schematic diagram of a flow chart of a method for obtaining a curved cylindrical body of a lung lobe image in one embodiment;

[0021] Fig.11 A schematic diagram of a process for obtaining a lung segmentation image in one embodiment;

[0022] Fig.12 is a schematic diagram of bronchial segmentation results in one embodiment;

[0023] Fig.13 is a schematic diagram of lung lobe segmentation results in one embodiment;

[0024] Fig.14 A schematic diagram of the bronchial topological structure in one embodiment;

[0025] Fig.15 is a schematic diagram of the recognition result of the root node of the vascular structure in one embodiment;

[0026] Fig.16 is a schematic diagram of a plane section of a lung lobe image in one embodiment;

[0027] Fig.17 is a schematic diagram of a curved section of a lung lobe image in one embodiment;

[0028] Fig.18 A schematic diagram of lung lobe plane section information in one embodiment;

[0029] Fig.19 A schematic diagram of lung lobe curved section information in one embodiment;

[0030] Fig. 20 A schematic diagram of a plane changing into a curved surface in an embodiment;

[0031] Fig.21 is a schematic diagram of a lung structure segmentation image in one embodiment;

[0032] Fig. 22 A schematic diagram of associated bronchial segments and bifurcation points in one embodiment;

[0033] Fig.23 This is a schematic diagram of LDA projection of a bronchial segment in one embodiment;

[0034] Fig.24 A schematic diagram of extracting a simplex boundary point set in one embodiment;

[0035] Fig.25 A schematic diagram of extracting a complex boundary point set in one embodiment;

[0036] Fig.26 A schematic diagram of plane reconstruction key point extraction in one embodiment;

[0037] Fig. 27 A schematic diagram of key point extraction for surface reconstruction in one embodiment;

[0038] Fig.28 A schematic diagram of PCA projection of a DFD point set in one embodiment;

[0039] Fig.29 A schematic diagram of extracting an inner DFD point set in an embodiment;

[0040] Fig.30 A schematic diagram of a registration point and a target point in one embodiment;

[0041] Fig.31 A schematic diagram of a process of reconstructing a curved cylindrical volume of a lung lobe image in one embodiment;

[0042] Fig.32 A schematic diagram of surface reconstruction in one embodiment;

[0043] Fig.33 A schematic diagram of an editing interaction process in an embodiment;

[0044] Fig.34 A schematic diagram of a process for extracting section information in an embodiment;

[0045] Fig.35 A schematic diagram of a lung lobe image plane section reconstruction process in one embodiment;

[0046] Fig.36 A schematic diagram of a lung segment algorithm flow chart in one embodiment;

[0047] Fig.37 is a structural block diagram of a lung segment image determination device in one embodiment;

[0048] Fig.38 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0050] The lung segment image determination method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other network servers. The server 104 obtains the lung scan image of the target object from the terminal 102, and further segments the lung scan image of the target object to obtain each bronchial segment segmentation image and each lobe segmentation image; the lung scan image is input into the vascular structure root node extraction model to obtain each vascular structure root node; each bronchial segment segmentation image and each vascular structure root node are input into the lobe image section reconstruction model to obtain the lobe image section corresponding to each lobe segmentation image; the corresponding lobe segmentation image is segmented according to each lobe image section to obtain each segment segmentation image. Among them, the terminal 102 can be but is not limited to an Internet of Things device and a portable wearable device. The server 104 can be implemented with an independent server or a server cluster consisting of multiple servers.

[0051] In one embodiment, Figure 2 As shown, a lung segment image determination method is provided, and the method is applied to Figure 1 The server in the example is used to illustrate the following steps:

[0052] Step 202, segmenting the lung scan image of the target object to obtain segmented images of each bronchial segment and segmented images of each lung lobe.

[0053] The target object may be a human or an animal, wherein the target object includes a region of interest, such as the liver, kidneys, lungs, etc. in the human body.

[0054] The lung scan image may be scan image data of a region of interest of the target object, wherein the scan image data includes but is not limited to magnetic resonance imaging data, X-ray scanning data, and computer tomography data.

[0055] The bronchial segment segmentation image may be a segmentation image of the bronchial segment of the lung obtained by performing image segmentation on the lung scan image.

[0056] The lung lobe segmentation image may be a segmentation image of the lung lobe obtained by performing image segmentation on the lung scan image.

[0057] Specifically, the server 104 obtains the lung scan image of the target object from the terminal 102, and further inputs the lung scan image of the target object into the image segmentation model, and obtains each bronchial segmentation image and each lung lobe segmentation image in the lung scan image by segmenting the lung scan image according to the structure of the lung bronchial segment and the lung lobe. The image segmentation model may be but is not limited to the nnUNet network. Each lung lobe segmentation image includes a left upper lobe segmentation image, a left lower lobe segmentation image, a right upper lobe segmentation image, a right middle lobe segmentation image, and a right lower lobe segmentation image. The bronchial segment segmentation image is as follows: Fig.12 As shown, in some embodiments, Fig.12 The main part of the bronchial segmentation image shown in FIG. 1 can be represented by color A (i.e., any color, such as yellow, red, green, blue, black, etc.), while other bronchial segment structures can be represented by colors different from color A. Fig.13 As shown, in some embodiments, Fig.13 Different lobes in the lung lobe segmentation image shown can be represented by two different colors.

[0058] Step 204: Input the lung scan image into the vascular structure root node extraction model to obtain each vascular structure root node.

[0059] Among them, the vascular structure root node extraction model can be a bronchial topology structure used to extract the vascular structure root nodes in the lung scan image, wherein the bronchial topology structure can be a structure that studies the geometric figures of the lung bronchi or some properties of the space that can remain unchanged after continuous changes in shape. It only considers the positional relationship of the lung bronchi without considering their shape and size.

[0060] The vascular structure root node may be a bifurcation point at the boundary between two associated bronchial segments in the lung structure.

[0061] Specifically, the server 104 generates a bronchial segmentation mask of the target object based on the lung scan image, and further uses an algorithm based on topological refinement to extract the centerline of the bronchial segmentation mask to obtain the bronchial centerline corresponding to the bronchial segmentation mask. By searching for the centerline endpoints, centerline bifurcation points and centerline continuous points of the bronchial centerline, as well as the information of the centerline endpoints, the information of the centerline bifurcation points and the information of the centerline continuous points, the root bronchus corresponding to the root node of the entire bronchus is extracted, and the root bronchus serves as the starting position for the subsequent extraction of the bronchial topological structure. Through the root bronchial position and the bronchial centerline, the "growth direction" of the bronchus in the bronchial topological structure is confirmed, that is, the edges of all bronchi should be directional, and when extracting the topological structure, it is necessary to first determine the extension direction starting from the root node of the root bronchus; when the extension direction is determined, based on the bronchial centerline, a traversal algorithm is used to search for the bronchial segmentation mask of the target object to obtain the bronchial topological structure, wherein the bronchial topological structure is such as Fig.14 As shown, in some embodiments, Fig.14 Different branches of the bronchial topology shown can be represented by different colors. When the bronchial topology corresponding to the bronchial segmentation mask is determined, all target common parent nodes in the bronchial topology are searched, that is, the target common parent nodes corresponding to the boundaries of the two associated bronchial segments in the lung structure, and further the endpoints with a larger number of connections are selected from the target common parent nodes as the root nodes of each vascular structure corresponding to the bronchial segmentation mask. Fig.15 is the recognition result of each vascular structure root node in the bronchial segmentation mask; wherein, Fig.15 The white circles in the figure are different root nodes of vascular structures.

[0062] Step 206: input each bronchial segment segmentation image and each vascular structure root node into the lung lobe image section reconstruction model to obtain the lung lobe image section corresponding to each lung lobe segmentation image.

[0063] The lung lobe image section reconstruction model may be a model that calculates data information used to generate lung lobe image sections.

[0064] The lung lobe image section may be a section used to further segment the lung lobe segmentation image.

[0065] Specifically, each bronchial segmentation image and each vascular structure root node are projected from three dimensions to two dimensions to obtain each two-dimensional bronchial segmentation image and each two-dimensional vascular structure root node. The lobe image sections include two different forms, namely, lobe image plane sections and lobe image curved sections. Therefore, the lobe plane section information corresponding to each lobe image plane section is calculated according to each two-dimensional bronchial segmentation image and each two-dimensional vascular structure root node, and the lobe curved section information corresponding to each lobe image curved section is calculated according to each two-dimensional bronchial segmentation image and each two-dimensional vascular structure root node. Finally, for the lobe plane section information corresponding to different lobe image plane sections, the lobe image plane sections are reconstructed using the plane equations constructed using the lobe plane section information; for the lobe curved section information corresponding to different lobe image curved sections, the lobe image curved sections are reconstructed using a layer-by-layer reconstruction method. The lobe image plane sections are as follows. Fig.16 As shown, label 1 is a segmented image of one lung segment, label 2 is another segmented image of another lung segment, and the section is a plane section of a lung lobe image. In one embodiment, the three are represented by two different colors. Fig.17 As shown, label 1 is one of the lung segment segmentation images, label 2 is another lung segment segmentation image, and the section is the section of the lung lobe image plane. In one embodiment, the three are represented by two different colors.

[0066] Step 208, segmenting the corresponding lung lobe segmentation image according to the slices of each lung lobe image to obtain each lung segmentation image.

[0067] The lung segmentation image may be an image of the lung segment after segmenting the lung lobe segmentation image, that is, information of the lung segment after segmentation of the lung scan image.

[0068] Specifically, for any lobe segmentation image, the lobe image section corresponding to the lobe segmentation image is used to segment it, and each unlabeled segmentation image corresponding to each lobe segmentation image can be obtained. According to real clinical data, under the lobe segmentation image as a constraint condition, each unlabeled segmentation image under the lobe segmentation image is labeled, and each segment segmentation image corresponding to each lobe segmentation image can be obtained.

[0069] In the above-mentioned lung segment image determination method, the lung scan image of the target object is segmented to obtain each bronchial segment segmentation image and each lung lobe segmentation image; the lung scan image is input into the vascular structure root node extraction model to obtain each vascular structure root node; each bronchial segmentation image and each vascular structure root node are input into the lung lobe image section reconstruction model to obtain the lung lobe image section corresponding to each lung lobe segmentation image; the corresponding lung lobe segmentation image is segmented according to each lung lobe image section to obtain each lung segmentation image.

[0070] By introducing the location information of pulmonary arteries and veins, the lung scan image is segmented and segmented to obtain each bronchial segmentation image and each lung lobe segmentation image. Then the lung scan image is topologically analyzed to extract bifurcation points at all levels for the lung lobe image section. Finally, the lung lobe image section is used to segment the corresponding lung lobe segmentation image to obtain each lung segmentation image, so that the distance from the lung lobe image section to the adjacent bronchial segmentation structure or accompanying artery is basically equal, thereby ensuring that the section passes through the intersegmental vein to the greatest extent, effectively improving the accuracy of lung segment segmentation in the lung image.

[0071] In one embodiment, Figure 3 As shown, each bronchial segment segmentation image and each vascular structure root node are input into the lobe image section reconstruction model to obtain the lobe image section corresponding to each lobe segmentation image, including:

[0072] Step 302: input each bronchial segment segmentation image and each vascular structure root node into a lung lobe image section reconstruction model to obtain lung lobe image section information corresponding to each lung lobe segmentation image.

[0073] The lung lobe image section information may be data information used to reconstruct the lung lobe image section.

[0074] Specifically, each bronchial segmentation image and each vascular structure root node are input into the lobe image section reconstruction model, and transformed into two-dimensional space through the linear discriminant algorithm (LDA) of the lobe image section reconstruction model to obtain the two-dimensional projection corresponding to each bronchial segmentation image; then, the two-dimensional bronchial segmentation image and the corresponding vascular structure root node are used to extract the lobe reconstruction boundary point set (data flow diagram point set (DFD)); finally, since the lobe image section information includes lobe plane section information and lobe curved section information, when the lobe image plane section needs to be reconstructed, the lobe plane section information is extracted from the lobe plane boundary point set in the lobe reconstruction boundary point set; and when the lobe image curved section needs to be reconstructed, the lobe curved section information is extracted from the lobe curved section point set in the lobe reconstruction boundary point set. The lobe plane section information is as follows: Fig.18The normal vectors and vascular structure root nodes (bifurcation points) shown in FIG. 1 are shown in FIG. 1 . In one embodiment, different structures are represented by two different colors. Fig.19 The direction vectors, vascular structure root nodes (bifurcation points) and curve point sets shown in FIG. 1 are, in one embodiment, represented by two different colors for different structures.

[0075] Step 304, reconstructing the lung lobe image sections corresponding to each lung lobe segmentation image according to the lung lobe image section information corresponding to each lung lobe segmentation image.

[0076] Specifically, before reconstructing the lung lobe image sections corresponding to each lung lobe segmentation image, the lung lobe plane section information and lung lobe curved section information can also be manually edited. Since the lung lobe plane section information cannot directly support drag and drop deformation operations, the lung lobe plane section information needs to be converted into lung lobe curved section information before interaction. Fig. 20 As shown: The lobe plane section information of the known plane I, that is, two points p1 and p2 and the normal vector Through p1 and p2, the normal vector Calculate plane II, then plane I and plane II are orthogonal and obtain a set of intersection points p i , calculate the normal vector of plane II Finally, the plane section information of the lung lobe is converted into the curved section information of the lung lobe, that is, the point set p i and direction vector Furthermore, the edit direction vector can be used to perform a rotation operation on a plane or a curved surface, or / and a translation operation on a point on the edited section, or / and a dragging and deformation operation on a point set on the edited section, to re-update the converted lung lobe curved section information. Fig.33 shown.

[0077] Since the lobe image sections include lobe image plane sections and lobe image curved sections, for each lobe image plane section, a plane parameter equation for reconstructing the lobe image plane section is constructed according to the lobe image section information, and the lobe image plane section is reconstructed according to the plane parameter equation; for each lobe image curved section, a three-dimensional cylindrical body for reconstructing the lobe image curved section is constructed according to the lobe image section information, and the lobe image curved section is reconstructed according to the three-dimensional cylindrical body.

[0078] In this embodiment, by using each bronchial segment segmentation image and each vascular structure root node to calculate the lobe image section information, and using the lobe image section information to reconstruct the lobe image section, the corresponding lobe image section can be determined according to the actual situation of different lungs, thereby improving the reconstruction accuracy of the lobe image section.

[0079] In one embodiment, Figure 4As shown, the lung scan image of the target object is segmented to obtain segmented images of each bronchial segment, including:

[0080] Step 402, performing vascular segmentation on the lung scan image to obtain a pulmonary artery segmentation image, a pulmonary vein segmentation image, and a bronchial segmentation image.

[0081] The pulmonary artery segmentation image may be a segmentation image of the pulmonary artery obtained by performing image segmentation on the lung scan image.

[0082] The pulmonary vein segmentation image may be a segmentation image of the pulmonary veins obtained by performing image segmentation on the lung scan image.

[0083] The bronchial segmentation image may be a segmentation image of the bronchial tubes in the lungs obtained by performing image segmentation on the lung scan image.

[0084] Specifically, the server 104 obtains a lung scan image of the target object from the terminal 102, imports it into a graphics processing module of the server 104, and generates a three-dimensional medical lung image composed of voxels; uses an image segmentation network (nnUNet network) to perform vascular segmentation on the three-dimensional medical lung image to obtain a pulmonary artery segmentation image, a pulmonary vein segmentation image, and a bronchial segmentation image. Fig.21 The pulmonary artery segmentation image, pulmonary vein segmentation image and bronchial segmentation image are obtained by segmenting the lung scan image using the image segmentation model, wherein the bronchial segmentation image can be represented by color A (i.e., any color, such as yellow, red, green, blue, black, etc.), and the pulmonary artery segmentation image and the pulmonary vein segmentation image can be represented by colors different from color A, and the two are different colors.

[0085] Step 404 , performing bronchial segmentation on the bronchial segmentation image according to the pulmonary artery segmentation image and the pulmonary vein segmentation image, to obtain bronchial segmentation images.

[0086] Specifically, the pulmonary artery segmentation image and the pulmonary vein segmentation image are used as position information, and the image segmentation network (nnUNet network) is used to segment the bronchial segmentation image to obtain segmentation images of each bronchial segment.

[0087] In this embodiment, by segmenting the lung scan image to obtain the bronchial segmentation image, the characteristics of the image segmentation model can be used to accurately obtain images of different lung tissues, providing precise data for the subsequent generation of lung lobe image sections.

[0088] In one embodiment, Figure 5As shown, each bronchial segment segmentation image and each vascular structure root node are input into the lobe image section reconstruction model to obtain the lobe image section information corresponding to each lobe segmentation image, including:

[0089] Step 502 : for any lung lobe segmentation image, select at least two target bronchial segmentation images from each bronchial segmentation image, and select a target vascular structure root node from each vascular structure root node.

[0090] The target bronchial segmentation image may be a bronchial segmentation image used to form one of the sections in the lung lobe segmentation image.

[0091] The target vascular structure root node may be a vascular structure root node used to form one of the sections in the lung lobe segmentation image.

[0092] Specifically, an association operation is performed on each bronchial segmentation map to obtain at least two target bronchial segmentation images, and a bifurcation point of each vascular structure root node is extracted to obtain a target vascular structure root node. The bronchial segmentation map has two categories, and the vascular structure root node has three categories; in one embodiment, Fig. 22 One of the situations shown, Fig. 22 The number of categories K = 3, the number of features n = 3, corresponding to the x, y, z coordinates of the three-dimensional image coordinate system. The enlarged radius of the bifurcation point is r = 2.5, that is, 2.5 pixel units.

[0093] Step 504: using the target vascular structure root node, determine a set of lung lobe reconstruction boundary line points from each target bronchial segment segmentation image.

[0094] The lung lobe reconstruction boundary point set may be a set of points that are equidistant from two different Euclidean distance field images.

[0095] Specifically, the linear discriminant algorithm (LDA) is used to transform the target vascular structure root node and each target bronchial segment segmentation image from three-dimensional space to two-dimensional space, and the two-dimensional projection corresponding to each target bronchial segment segmentation image is obtained, and two Euclidean distance field images are generated according to the two-dimensional projection corresponding to each target bronchial segment segmentation image. The target vascular structure root node is used as a reference point, and the lobe reconstruction boundary point set is extracted on the curve with equal distance to the Euclidean distance field image.

[0096] Step 506, generating lung lobe image section information according to the lung lobe reconstruction boundary line point set.

[0097] Specifically, since the lobe reconstruction boundary point set includes the lobe plane boundary point set and the lobe curved surface boundary point set, and correspondingly, the lobe image section information includes the lobe plane section information and the lobe curved surface section information. For the lobe plane section information, the lobe plane section information is obtained by performing a linear discriminant algorithm (LDA) inverse transformation on the target vascular structure root node in the lobe plane boundary point set and any lobe plane boundary point that satisfies the constructed plane parameter equation; for the lobe curved surface section information, the lobe curved surface section information is obtained by performing a linear discriminant algorithm (LDA) inverse transformation on the target vascular structure root node in the lobe plane boundary point set and the sampling points from the remaining lobe curved surface boundary points.

[0098] In this embodiment, the lobe reconstruction boundary point set is calculated based on each target bronchial segment segmentation image and the target vascular structure root node, and the lobe reconstruction boundary point set is further used to generate lobe image section information, which can ensure that the section passes through the vascular structure root node and passes through the inter-segment veins to the greatest extent.

[0099] In one embodiment, Figure 6 As shown, with the target vascular structure root node, the pulmonary lobe reconstruction boundary point set is determined from each target bronchial segment segmentation image, including:

[0100] Step 602 : Perform two-dimensional projection on each target bronchial segment segmentation image and the target vascular structure root node to obtain a bronchial segment projection image.

[0101] The bronchial segment projection image may be an image obtained by projecting the target bronchial segment segmentation image from three dimensions to two dimensions.

[0102] Specifically, since the bronchial segmentation map has two categories, and the vascular structure root node has three categories, the three-class linear discriminant algorithm (LDA) is used to reduce the dimension of the two types of bronchial segmentation maps and the vascular structure root nodes, and the bronchial segment projection images in the two-dimensional space with the minimum intra-class variance and the maximum inter-class variance of the two types of bronchial segments are obtained, and the coordinate system of the bronchial segment projection image is converted into the image coordinate system.

[0103] The linear discriminant algorithm projection matrix is ​​LM, the bronchial segment segmentation map and the vascular structure root node are in the three-dimensional image coordinate system W3, and the plane coordinate system W2' of the bronchial segment projection image is:

[0104] W2 ′ =W3·LM

[0105] The LM calculation process is as follows:

[0106] The Rayleigh quotient formula is defined as:

[0107]

[0108] Among them, u T is a unit vector, S w is the intra-class scatter matrix, S b is the inter-class scatter matrix, and the calculation formula is as follows:

[0109]

[0110]

[0111]

[0112] The dataset of the k-th sample is D k , m k is the mean vector of the k-th class of samples. k It is a vector composed of all features of the k-th class samples, and each feature is the x, y, and z coordinates of the three-dimensional image coordinate system.

[0113] when When , the eigenvectors w1 and w2 corresponding to the two largest eigenvalues ​​of LM are calculated. The matrix LM composed of vectors w1 and w2 can maximize the Rayleigh quotient J(u), thereby obtaining the maximum projection of LDA.

[0114] The plane coordinate system W2' of the bronchial segment projection image is linearly transformed to the image coordinate system W2 of the bronchial segment projection image, and the transformation coefficients L and b are:

[0115] W2=L·W2'+b

[0116] After the image coordinate system W2 of the bronchial segment projection image is obtained, image processing operations can be performed on the mapped bronchial segment projection image.

[0117] The projection images of the bronchial segments and the projection of the root nodes of the target vascular structures are divided into two cases. Fig.23 As shown, in some embodiments, Fig.23 The projection images of different bronchial segments and the root nodes of the vascular structure shown can be represented by two different colors.

[0118] like Fig.23 The left side of FIG. 1 shows the first case, bronchial segment projection sub-image 1 (bronchial segment 1), bronchial segment projection sub-image 2 (bronchial segment 2) and the target vascular structure root node (bifurcation point) sub-projection. This case is a common case, that is, it is used to extract the boundary line point set of the lobe plane;

[0119] like Fig.23The right side of is shown in the first case, bronchial segment projection sub-image 1 (bronchial segment 1), local bronchial trunk, target vascular structure root node (bifurcation point) sub-projection, that is, used to extract the boundary line point set of the lobe surface. This case is mainly for the dorsal segment of the left lower lobe and the dorsal segment of the right lower lobe. At this time, the bronchial segment associated with the dorsal segment is the trunk, and the associated bronchial segments that need to extract the boundary line are still the four basal segments of inner, front, outer and posterior. At this time, these four basal segments need to be set to the same label and projected onto the plane obtained by linear discriminant algorithm dimensionality reduction of bronchial segment projection sub-image 1 (bronchial segment 1), the trunk and the target vascular structure root node (bifurcation point) sub-projection.

[0120] Step 604: input the bronchial segment projection image into a boundary point set extraction model to obtain a lung lobe reconstruction boundary point set.

[0121] Among them, the boundary point set extraction model can be a point set extraction algorithm based on a data flow graph.

[0122] Specifically, the bronchial segment projection image is input into the boundary point set extraction model, and according to the projection of the bronchial segment projection image and the root node of the target vascular structure, a sub-model for extracting the lobe plane boundary point set or a sub-model for extracting the lobe curved surface boundary point set is selected from the boundary point set extraction model; based on the extraction of the lobe plane boundary point set, the projection of the root node of the target vascular structure is used as a constraint condition, and the bronchial segment projection image is input into the sub-model of the plane boundary point set to obtain the lobe plane boundary point set; based on the extraction of the lobe curved surface boundary point set, the projection of the root node of the target vascular structure is used as a constraint condition, and the bronchial segment projection image is input into the sub-model of the curved surface boundary point set to obtain the lobe curved surface boundary point set.

[0123] In this embodiment, by performing two-dimensional projection on the segmentation images of each target bronchial segment and the root node of the target vascular structure and then calculating the lobe reconstruction boundary point set, the distance from the section to the adjacent bronchial segment or accompanying artery can be basically equal, thereby improving the subsequent segmentation accuracy.

[0124] In one embodiment, Figure 7 As shown, the pulmonary lobe reconstruction boundary line point set includes the pulmonary lobe plane boundary line point set and the pulmonary lobe curved surface boundary line point set; the bronchial segment projection image is input into the boundary line point set extraction model to obtain the pulmonary lobe reconstruction boundary line point set, including:

[0125] Step 702: perform convex hull extraction on the bronchial segment projection image to obtain a bronchial segment convex hull image.

[0126] The bronchial segment convex hull image may be an image obtained by extracting the bronchial segment projection image using a convex hull algorithm.

[0127] Specifically, the convex hull algorithm is respectively run on the two bronchial segment projection images on the two-dimensional image matrix W2 to implement convex hull extraction (Convex Hull) to obtain the bronchial segment convex hull image.

[0128] Step 704: Generate a first plane distance field image and a second plane distance field image according to the bronchial segment convex hull image.

[0129] The first plane distance field image and the second plane distance field image may be first Euclidean plane distance field function imaging and second Euclidean plane distance field function imaging.

[0130] Specifically, after the bronchial segment convex hull image is flood filled, Fig.24 The schematic diagram of "convex hull extraction and flood filling" in , the fast Euclidean distance algorithm (SEDT) is used to calculate the Euclidean distance field images of the two convex hull background to foreground (BackGround To ForeGround, denoted as BF) respectively, and the first plane distance field image is obtained, as shown in Fig.24 A schematic diagram of the "convex hull 1B-F distance field" in , and a second plane distance field image, such as Fig.24 Diagram of the "Convex Hull 2B-F Distance Field" in .

[0131] Step 706: fuse the first plane distance field image and the second plane distance field image to obtain a set of lung lobe plane boundary line points.

[0132] Specifically, the point sets of the first plane distance field image and the second plane distance field image that are approximately equal are taken as the points on the boundary line of each target bronchial segment segmentation image, and are recorded as the lobe plane boundary line point set, such as Fig.24 Schematic diagram of "Extracting DFD boundary point set" in .

[0133] Alternatively, in step 708, a first curved surface distance field image and a second curved surface distance field image are generated according to the bronchial segment projection image.

[0134] The first curved surface distance field image and the second curved surface distance field image may be first Euclidean surface distance field function imaging and second Euclidean surface distance field function imaging.

[0135] Specifically, due to the complex structures of the back segment and other basal segments, the extracted convex hulls may have intersections, so it is impossible to obtain the boundary information of the data flow graph with strong constraints through the convex hull. By not extracting the convex hull and filling the other basal segments on the bronchial segment projection sub-image 1 (bronchial segment 1) on the two-dimensional image matrix, the Euclidean distance field image from the background to the foreground is directly calculated by the fast Euclidean distance algorithm (SEDT), and the first surface distance field image is obtained, as shown in FIG. Fig.25Schematic diagram of the "Bronchi segment 1B-F distance field" in , and the second surface distance field image, as shown in Fig.25 Schematic diagram of the “Bronchi segment 2B-F distance field” in FIG.

[0136] Step 710: fuse the first curved surface distance field image and the second curved surface distance field image to obtain a lung lobe curved surface boundary line point set.

[0137] Specifically, the point sets of the first curved surface distance field image and the second curved surface distance field image that are approximately equal are taken as the points on the boundary line of each target bronchial segmentation image, and are recorded as the lung lobe curved surface boundary line point set, such as Fig.25 Schematic diagram of "Extracting DFD boundary point set" in .

[0138] Among them, the Euclidean distance algorithm (SEDT) algorithm process is as follows:

[0139] The following table shows an arbitrary 3×3 area in the background area of ​​the image.

[0140] <![CDATA[p1]]> <![CDATA[p2]]> <![CDATA[p3]]> <![CDATA[p4]]> <![CDATA[q(p5)]]> <![CDATA[p6]]> <![CDATA[p7]]> <![CDATA[p8]]> <![CDATA[p9]]>

[0141] Among them, p i and q represent the coordinates of the current pixel. In a two-dimensional image, the image composed of the minimum Euclidean distance from the coordinates of all background areas to the foreground is called a distance field image, denoted as DF (Distance Field) image. i The Euclidean distance is h(q,p i ), the value is:

[0142] h(q,p i )=0(i=5)

[0143] h(q,p i )=1(i=2,4,6,8)

[0144]

[0145] Initialize the pixel value of all DF images to 10000, then p i The DF values ​​of and q are:

[0146] f(p i )=10000,f(q)=10000·

[0147] Traverse from the upper left corner to the lower right corner:

[0148] <![CDATA[p1]]> <![CDATA[p2]]> <![CDATA[p3]]> <![CDATA[p4]]> q

[0149] Then the state equation in the neighborhood is:

[0150] f(q)=min(f(q),f(q)+h(q,p i ))(i=1,2,3,4)

[0151] After the traversal is completed, the forward DFB distance field image is obtained. To ensure that all DFB values ​​are minimum, it is necessary to traverse from the lower right corner to the upper left corner:

[0152] q <![CDATA[p6]]> <![CDATA[p7]]> <![CDATA[p8]]> <![CDATA[p9]]>

[0153] Then the state equation in the neighborhood is:

[0154] f(q)=min(f(q),f(q)+h(q,p i ))(i=6,7,8,9)

[0155] After two iterations of DF distance field calculation, the DF image with the smallest Euclidean distance from the background area to the foreground mask of the whole image is finally obtained. DF1 ,I DF2 Perform a subtraction operation to obtain the coordinate point where the absolute value of the difference val is less than the threshold thr, that is:

[0156] val=|I DF1 -I DF2 | <thr

[0157] Where thr = 0.5. If thr is too large, too many points will be detected and they will not be concentrated; if thr is too small, too few points will be detected and it will be impossible to extract enough boundary point set information.

[0158] All coordinate points corresponding to val that meet the conditions constitute the DFD (Distance Field Difference) boundary point set I DFD .

[0159] In this embodiment, by using two different planar Euclidean distance field images to determine the plane boundary line point set of the pulmonary lobes, and two different curved Euclidean distance field images to determine the curved boundary line point set of the pulmonary lobes, the spatial structure of different target bronchial segments in the bronchial segment projection image can be accurately described, thereby facilitating the acquisition of the boundary line point set.

[0160] In one embodiment, Figure 8 As shown, the pulmonary lobe image section information includes pulmonary lobe plane section information and pulmonary lobe curved section information; the pulmonary lobe image section information is generated according to the pulmonary lobe reconstruction boundary line point set, including:

[0161] Step 802, determining two lobe plane reconstruction key points from the lobe plane boundary point set.

[0162] The lobe plane reconstruction key point may be a lobe plane boundary point used to construct a reconstruction plane equation.

[0163] Specifically, two lobe plane boundary points on the lobe plane boundary point set need to be found for three-dimensional plane reconstruction, one of which is the two-dimensional projection point of the root node of the target vascular structure, and the other point is calculated in the lobe plane boundary point set. The calculation process is as follows:

[0164] Traversing the boundary point set of the lung lobe plane I DFD , returns the point P where val is the smallest a .

[0165] P a As the center, I DFD All points P in i The PCA algorithm is used to obtain the inner point set I of the two target bronchial segment segmentation images. DFDin .

[0166] The three-dimensional coordinates of the root node of the target vascular structure are marked as P r3D , the projection matrix is ​​LM, and the projection coordinates of the root node of the target vascular structure are obtained:

[0167] P r2D =P r3D ·LM

[0168] make

[0169] P1=P r2D

[0170] Traverse the inner point set I of the boundary point set of the lobe plane DFDin , find the first point P that satisfies the following conditions b :

[0171]

[0172] When δ is too large, it will cause P a and P B The two points are too dispersed, causing the line connecting the two points to interfere with the segmented image of the target bronchial segment at the distal end;

[0173] When δ is too small, P A and P b The line connecting two points is not enough to represent the entire inner point set I DFdIN Direction of movement.

[0174] Therefore, let δ=5.0 here.

[0175] Calculate P a and P b The midpoint between the two points is denoted as P2, such as Fig.26 As shown, that is:

[0176] P2=(p a +p b ) / 2

[0177] Step 804, perform dimensional transformation on the key points of each lobe plane reconstruction to obtain lobe plane section information.

[0178] Specifically, the coordinate points P1 and P2 of the two-dimensional lung lobe plane reconstruction key points are linearly combined to obtain a third point P3. The calculation process is as follows:

[0179] P3.x=(P1.x+P2.x) / 2-(P1.y+P2.y) / 6

[0180] P3.y=(P1.y+P2.y) / 2-(P1.x+P2.x) / 6

[0181] Through the above calculation, it is ensured that the three points P1, P2, and P3 are not collinear.

[0182] Transform the image coordinate system back to the projection plane coordinate system, that is:

[0183] The above three points are linearly transformed from the image coordinate system W2 to the image coordinate system W2', with the transformation coefficients L and b, then:

[0184] W2'=L -1 (W2-b)

[0185] After LDA inverse transformation, P1, P2, and P3 will obtain three three-dimensional coordinate points.

[0186] P 2D =[P1, P2, P3] T

[0187] P 3D =P 2D ·LM -1

[0188] The plane and plane normal vector are calculated using the three points P1, P2, and P3, and the three-dimensional coordinates of the normal vector and the bifurcation point P1 are output as the lobe plane section information, such as Fig.18 Normal vectors and vascular structure root nodes (bifurcation points) are shown.

[0189] Alternatively, in step 806, the boundary point set of the lung lobe surface is fitted to obtain a boundary point set fitting curve.

[0190] The boundary point set fitting curve may be a curve obtained by fitting the boundary points of each lung lobe surface.

[0191] Specifically, it is necessary to find a point set on the two-dimensional plane curve in the boundary point set of the lobe surface, first use the PCA algorithm to obtain the inner point sets of the two bronchial segments, and then mark the inner point sets in an ordered manner, and then perform Bezier curve fitting on the screened ordered point sets to obtain the boundary point set fitting curve.

[0192] Step 808, sampling is performed from the fitting curve of the boundary point set to obtain a number of lung lobe surface reconstruction key points.

[0193] The key points for reconstruction of the lung lobe surface may be boundary points of the lung lobe surface used to construct a three-dimensional cylindrical body.

[0194] Specifically, the fitted curve is then sampled, and the sampled ordered point set is output for three-dimensional surface reconstruction. DFD , returns the point P where val is the smallest a .

[0195] P a As the center, I DFD All points P in i Divided into two categories, the PCA algorithm is used to obtain the inner point set I of the two target bronchial segment segmentation images DFDin .

[0196] The three-dimensional coordinates of the root node of the target vascular structure are marked as P r3D , the projection matrix is ​​LM, and the projection coordinates of the root node of the target vascular structure are obtained:

[0197] P r2D =P r3D ·LM

[0198] P r2D Add to inner point set I DFDin and I DFBin All points in P r2D Mark the first point in an orderly manner and perform n-order Bezier curve fitting. The given points are P0, P1, P2, ..., P n , then the point B(t) on the Bezier curve can be calculated by the following formula:

[0199]

[0200] It can be seen from the calculation formula that the fitting curve is a set of discrete points.

[0201] represents the Pascal's triangular coefficient matrix;

[0202]

[0203] P iRepresents coordinate points, the number of points is N;

[0204] n represents the order of fitting, which is obtained from N-1;

[0205] t represents the sampling interval, which is obtained by i / (n+1).

[0206] After n-order Bezier curve fitting, several lung lobe surface reconstruction key points of the fitting curve sampling points are output, denoted as I DFDin 'like Fig. 27 The first picture from the left is the lung lobe surface boundary point set (DFD point set), such as Fig. 27 The second picture from the left is the point set of the inner side of two bronchial segments obtained by PCA algorithm (PCA classification), as shown in Fig. 27 The third picture from the left is the result of filtering the inner point set (filtering the inner point set), such as Fig. 27 The fourth picture from the left shows several key points of lung lobe surface reconstruction (Bazier curve fitting sampling point set).

[0207] Determine the boundary point set of the lung lobe surface I DFD All points P in i Is it x-direction or y-direction, let:

[0208] y min =min(P i .y), y max =max(P i .y)

[0209] x min =min(P i .x), x max =max(P i .x)

[0210] if:

[0211] |y max -y min |>|x max -x min |:P i It is y direction,

[0212] |y max -y min |≤|x max -x min |:P i It is the x direction,

[0213] When P i is y-direction, then all points P i Sort by y, and put P i .y>P a.y points are classified as category I, P i .y≤P a .y is classified as category II, and the points of the two categories are assigned labels 1 and 2 respectively;

[0214] When P i is the direction of x, then all points P i Sort by x, and put P i .x>P a .x point is classified as category I, P i .x≤P a .x is classified as category II, and the points of the two categories are assigned labels 1 and 2 respectively.

[0215] The PCA algorithm is used to classify the ordered points in categories I and II. All points are projected into one-dimensional space through an unsupervised method. The projection matrix is ​​recorded as PM, and then:

[0216] I DFD1D =I DFD PM

[0217] The PM calculation process is as follows:

[0218] Define the variance of sample X after projection on the plane:

[0219] σ 2 =w T Sw

[0220]

[0221] X=[x1 T , x2 T , …, x n T ] T , (n=1,2)

[0222] Among them, S is the covariance matrix of the sample, w is the eigenvector of S, all eigenvalues ​​of S are calculated and sorted by singular value decomposition (SVD), and then the eigenvector corresponding to the eigenvalue with the largest q is selected, which can ensure the variance σ 2 Maximum, and get the projection matrix PM.

[0223] Projecting from two dimensions to one dimension, q=2, the eigenvectors are w1 and w2 respectively, then we have.

[0224] PM=w1

[0225] Get I DFD1D The label of the point set is marked with label 1 or label 2, such as Fig.28As shown in the figure, the minimum and maximum values ​​of the one-dimensional point set with pixel values ​​of label 1 and label 2 are calculated respectively, which are recorded as X 1min , X 1max , X 2min , X 2max .

[0226] Calculate the boundary points M of the two types of point sets:

[0227] if|X 1min -X 2max |<|X 2min -X 1max |:M=(X 1min +X 2max ) / 2

[0228] if|X 1min -X 2max |≥|X 2min -X 1max |:M=(X 2min +X 1max ) / 2

[0229] Map point M back to two-dimensional space to get point A, and calculate point B through point A and projection vector w2:

[0230] A=M·PM -1

[0231] B=A+w2 T

[0232] Then the straight line AB divides the projection image of the bronchial segment in the two-dimensional plane into two parts.

[0233] Calculate the equation of the line Ax+By+C=0 through points A and B, substitute each point on the projection image of the bronchial segment into the equation, and count the number of points greater than 0 and less than 0, respectively, and record them as v1 and v2:

[0234] Ax+By+C>0:v1(p i )

[0235] Ax+By+C<0:v2(p i )

[0236] like Fig.29 As shown, if v1>v2, then I DFD Substitute the midpoint set into the equation. If Ax+By+C>0, then the inner point set I DFDin On the contrary, if v1<v2, then I DFD Substitute the midpoint set into the equation. If Ax+By+C<0 is satisfied, then the inner point set I DFDin .

[0237] Step 810, perform dimension conversion on the reconstructed key points of each lung lobe surface to obtain lung lobe surface section information.

[0238] Specifically, the first point P1 and the last point P n The two coordinate points are calculated by linear combination to obtain the Pth n+1 The calculation process is as follows:

[0239] P n+1 .x=(P1.x+P n .x) / 2-(P1.y+P n .y) / 6

[0240] P n+1 .y=(P1.y+P n .y) / 2-(P1.x+P n .x) / 6

[0241] After the n+1 two-dimensional coordinate points are inversely transformed by LDA, n+1 three-dimensional coordinate points will be obtained.

[0242] P 2D =[P1, P2, ... P n ] T

[0243] P 3D =P 2D ·LM -1

[0244] Using P1, P n , P n+1 The plane equation is calculated from the three points, and then the other n-2 points that are approximately in the same plane are constrained to the plane, and the plane normal vector is calculated. The normal vector is used as the direction vector of the surface. n+1 The three-dimensional coordinates of n points outside the point are output as the lobular surface section information. In one embodiment, the process of extracting the section information is as follows: Fig.34 shown.

[0245] In this embodiment, by performing LDA inverse transformation on the key points of each lung lobe reconstruction, the cross-sectional information of the lung lobe is obtained, and the key points of each lung lobe reconstruction can be inversely transformed from the image coordinate system to the projection plane coordinate system, so that low-dimensional data is converted into high-dimensional data, making it easier to apply the cross-sectional information of the lung lobe.

[0246] In one embodiment, Fig. 9 As shown, the lung lobe image sections include lung lobe image plane sections and lung lobe image curved sections; according to the lung lobe image section information corresponding to each lung lobe segmentation image, the lung lobe image sections corresponding to each lung lobe segmentation image are reconstructed, including:

[0247] Step 902, determining the lung lobe image plane and constructing a model according to the lung lobe image section information.

[0248] The lung lobe image plane construction model may be a plane parameter equation used to construct a section of the lung lobe image plane.

[0249] Specifically, according to the root node coordinates and normal vector of the target vascular structure in the section information of the lung lobe image, the plane parameter equation is calculated by the point method, that is, the lung lobe image plane construction model is constructed.

[0250] Step 904, constructing a lung lobe image plane section corresponding to the lung lobe segmentation image according to the lung lobe image plane construction model.

[0251] Specifically, a model (plane parameter equation) is constructed according to the lung lobe image plane to generate a lung lobe image plane section corresponding to the lung lobe segmentation image, wherein the lung lobe image plane section divides the lung lobe segmentation image into two parts, such as Fig.16 As shown, label 1 is one of the lung segmentation images, label 2 is another lung segmentation image, and the section is the section of the lung lobe image plane. In one embodiment, the three are represented by two different colors. , and two unlabeled lung segmentation images 1 and 2 are obtained.

[0252] Alternatively, in step 906, layer-by-layer reconstruction is performed based on the lung lobe curved section information to obtain a lung lobe image curved cylindrical volume.

[0253] The lung lobe image curved cylindrical body may be a three-dimensional cylindrical body used to generate a lung lobe image curved section.

[0254] Specifically, according to the lung lobe curved section information, layer-by-layer three-dimensional reconstruction is performed in a specified direction to obtain a lung lobe image curved cylindrical volume.

[0255] Step 908, selecting the intersection of the lung lobe image curved cylinder and the lung lobe segmentation image to obtain a lung lobe image curved section.

[0256] Specifically, the intersection between the lung lobe image surface cylinder and the lung lobe segmentation image is taken to obtain the lung lobe image surface section, such as Fig.17 As shown, label 1 is one of the lung segmentation images, label 2 is another lung segmentation image, and the section is the section of the lung lobe image plane. In one embodiment, the three are represented by two different colors to obtain two unlabeled lung segmentation images 1 and 2.

[0257] In this embodiment, by using the section information of the lung lobe to reconstruct the lung lobe image section corresponding to the lung lobe segmentation image, it can be ensured that the interface passes through the root node of the vascular structure or fits the inter-segmental vein, thereby reducing the deviation between the subsequent lung segmentation image and the clinical scene.

[0258] In one embodiment, Fig.10 As shown, according to the lung lobe curved section information, layer-by-layer reconstruction is performed to obtain a lung lobe image curved cylindrical body, including:

[0259] Step 1002, perform two-dimensional projection on the pulmonary lobe curved section information to obtain a pulmonary lobe curved section projection point set.

[0260] The lung lobe curved section projection point set may be a set of projection points obtained by projecting lung lobe curved section information from three dimensions to two dimensions.

[0261] Specifically, calculate n P in the lung lobe surface section information I The average point coordinates of the points are recorded as point G. Taking point G as the base point, the normal vector is used to calculate the spatial point L of the line where the normal vector is located. Obviously, there is a vector LG perpendicular to the plane where the n points are located, and P1 is recorded as point C. Using the three points C, G, and L that are orthogonal in space, an arbitrary coordinate system W in space is constructed, and the orthogonal coordinate system W is rotated and translated to a direction parallel to the image coordinate system, and the coordinates C1, G1, and L1 corresponding to the three points in the new coordinate system W1 are calculated, as shown in Fig.30 Schematic diagram of the registration points and target points shown. The two sets of coordinates (C, G, L) and (C1, G1, L1) are used for homogeneous linear transformation to calculate the RT transformation matrix T. The n points are forward mapped to the coordinate system W1 through the registration matrix T, and in the W1 coordinate system, the n three-dimensional coordinate points are projected in the z direction: (x, y, z) → (x, y), and a set of projection points of the lung lobe surface section with n two-dimensional coordinate points is obtained.

[0262] Step 1004, generating a lung lobe surface section projection curve according to the lung lobe surface section projection point set.

[0263] The lobe surface section projection curve may be a curve obtained by connecting the lobe surface section projection point set.

[0264] Specifically, for a set of n ordered two-dimensional coordinate points of the pulmonary lobe section projection point set, Bresenham's Line Algorithm is used to connect multiple points to obtain a curve, namely, the pulmonary lobe section projection curve, and rays are extended from both ends of the pulmonary lobe section projection curve to the boundary of the two-dimensional image.

[0265] Step 1006, reconstructing layer by layer according to the projection curve of the lung lobe surface section to obtain a lung lobe image surface cylindrical volume.

[0266] Specifically, Fig.31As shown, the coordinates of the projection curve point set of the lung lobe surface section are obtained, and the points on the current curve and the four corner points are classified according to the intersection between the curve and the boundary. The closed loop formed by multi-point connection is obtained, and the closed loop is filled with water; finally, the two-dimensional flood-filled image is reconstructed layer by layer in the z direction to obtain a three-dimensional lung lobe image surface cylinder. Further, as Fig.32 The schematic diagram of surface reconstruction shown in FIG. 1 is to use the three-dimensional lung lobe image surface cylinder to obtain the inverse matrix T' of the registration matrix T, and to reversely map the lung lobe image surface cylinder to obtain the lung lobe image surface section of the cylinder in the original orthogonal coordinate system W space. The lung lobe image plane section reconstruction process is as follows: Fig.35 shown.

[0267] In this embodiment, by using the lung lobe surface section information to perform layer-by-layer reconstruction to construct a lung lobe image surface cylindrical body, it is possible to solve the intersection of the lung lobe image surface cylindrical body and the lung lobe segmentation image, thereby improving the accuracy of reconstructing the lung lobe image surface section.

[0268] In one embodiment, Fig.11 As shown, the corresponding lung lobe segmentation images are segmented according to the sections of each lung lobe image to obtain the segmentation images of each lung segment, including:

[0269] Step 1102, segmenting the lung lobe segmentation image according to the plane section of the lung lobe image to obtain each unlabeled lung segmentation image.

[0270] The unlabeled lung segmentation image may be a lung segmentation image that has been segmented but has not been labeled.

[0271] Specifically, the lung lobe segmentation image is used as a constraint condition, and at least one lung lobe image plane section in the lung lobe segmentation image is used to segment the lung lobe segmentation image to obtain each unlabeled lung segmentation image.

[0272] Step 1104, labeling each unlabeled lung segmentation image according to the positional relationship between the lung segmentation image and the lung segmentation image to obtain each lung segmentation image.

[0273] The positional relationship of the lung segment images may be the positional relationship between the lung segments in a real situation.

[0274] Specifically, the lung segment image position relationship of each unlabeled lung segmentation image in the real scene is known, for example, the relative lung segment image position relationship of unlabeled lung segmentation image 1 and unlabeled lung segmentation image 2, assuming that the lung segment image position relationship is an up-down relationship. Combine the equation and the normal vector direction to determine the lung segment orientation associated with the lung lobe segmentation, mark each unlabeled lung segmentation image, and obtain each lung segmentation image.

[0275] Alternatively, in step 1106, the lung lobe segmentation image is segmented according to the curved section of the lung lobe image to obtain each unlabeled lung segmentation image.

[0276] Specifically, the lung lobe segmentation image is used as a constraint condition, and at least one lung lobe image curved section in the lung lobe segmentation image is used to segment the lung lobe segmentation image to obtain each unlabeled lung segmentation image.

[0277] Step 1108, traversing the positional relationship of the lung segment images of the lung lobe segmentation image, marking each unlabeled lung segmentation image according to the traversal result, and obtaining each lung segmentation image.

[0278] Specifically, the lung segment image position relationship of each unlabeled lung segmentation image in the real scene is known, for example: the relative lung segment image position relationship of unlabeled lung segmentation image 1 and unlabeled lung segmentation image 2, assuming that the lung segment image position relationship is an up-down relationship, then traverse from top to bottom. If unlabeled lung segmentation image 1 is encountered first, then unlabeled lung segmentation image 1 is assigned label 1, and unlabeled lung segmentation image 2 is assigned label 2; if unlabeled lung segmentation image 2 is encountered first, then unlabeled lung segmentation image 2 is assigned label 1, and unlabeled lung segmentation image 1 is assigned label 2. Various different lung segment image position relationships are deduced in turn, and each unlabeled lung segmentation image is labeled to obtain each lung segmentation image. In one embodiment, a schematic flow chart of the lung segment algorithm for the lung segment image determination method is shown as follows. Fig.36 shown.

[0279] In this embodiment, by marking each unlabeled lung segmentation image according to the lung segment image position relationship in the real scene, it can be ensured that the lung segmentation image matches the data in the real scene and the error of the lung segmentation image is reduced.

[0280] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0281] Based on the same inventive concept, the embodiment of the present application also provides a lung segment image determination device for implementing the lung segment image determination method involved above. The implementation solution provided by the device to solve the problem is similar to the implementation solution recorded in the above method, so the specific limitations in one or more lung segment image determination device embodiments provided below can refer to the above limitations on a lung segment image determination method, and will not be repeated here.

[0282] In one embodiment, Fig.37 As shown, a lung segment image determination device is provided, comprising: a lung image segmentation module 3702, a vascular structure root node acquisition module 3704, a lung lobe image section acquisition module 3706 and a lung segment segmentation image acquisition module 3708, wherein:

[0283] The lung image segmentation module 3702 is used to segment the lung scan image of the target object to obtain segmented images of each bronchial segment and each lung lobe;

[0284] A vascular structure root node obtaining module 3704 is used to input the lung scan image into the vascular structure root node extraction model to obtain each vascular structure root node;

[0285] A lung lobe image section obtaining module 3706 is used to input each bronchial segmentation image and each vascular structure root node into a lung lobe image section reconstruction model to obtain a lung lobe image section corresponding to each lung lobe segmentation image;

[0286] The lung segmentation image obtaining module 3708 is used to segment the corresponding lung lobe segmentation image according to the image section of each lung lobe to obtain each lung segmentation image.

[0287] In one embodiment, the lobe image section obtaining module 3706 is also used to input each bronchial segmentation image and each vascular structure root node into the lobe image section reconstruction model to obtain the lobe image section information corresponding to each lobe segmentation image; and reconstruct the lobe image section corresponding to each lobe segmentation image based on the lobe image section information corresponding to each lobe segmentation image.

[0288] In one embodiment, the lung image segmentation module 3702 is also used to perform vascular segmentation on the lung scan image to obtain a pulmonary artery segmentation image, a pulmonary vein segmentation image and a bronchial segmentation image; based on the pulmonary artery segmentation image and the pulmonary vein segmentation image, the bronchial segmentation image is segmented into bronchial segments to obtain segmentation images of each bronchial segment.

[0289] In one embodiment, the lobe image section obtaining module 3706 is also used to select at least two target bronchial segmentation images from each bronchial segmentation image and select a target vascular structure root node from each vascular structure root node for any lobe segmentation image; determine the lobe reconstruction boundary point set from each target bronchial segment segmentation image using the target vascular structure root node; and generate the lobe image section information according to the lobe reconstruction boundary point set.

[0290] In one embodiment, the lung lobe image section obtaining module 3706 is also used to perform two-dimensional projection on each target bronchial segment segmentation image and the target vascular structure root node to obtain a bronchial segment projection image; the bronchial segment projection image is input into a demarcation line point set extraction model to obtain a lung lobe reconstruction demarcation line point set.

[0291] In one embodiment, the module 3706 for obtaining the section of the lung lobe image is further used to perform convex hull extraction on the bronchial segment projection image to obtain the bronchial segment convex hull image; generate a first plane distance field image and a second plane distance field image according to the bronchial segment convex hull image; fuse the first plane distance field image and the second plane distance field image to obtain a lung lobe plane boundary line point set; or generate a first curved surface distance field image and a second curved surface distance field image according to the bronchial segment projection image; fuse the first curved surface distance field image and the second curved surface distance field image to obtain a lung lobe curved surface boundary line point set.

[0292] In one embodiment, the module 3706 for obtaining the section of the lobe image is also used to determine two lobe plane reconstruction key points from the lobe plane boundary line point set; perform dimension conversion on each lobe plane reconstruction key point to obtain lobe plane section information; or, perform fitting on the lobe surface boundary line point set to obtain a boundary line point set fitting curve; perform sampling from the boundary line point set fitting curve to obtain a number of lobe surface reconstruction key points; perform dimension conversion on each lobe surface reconstruction key point to obtain lobe surface section information.

[0293] In one embodiment, the lobe image section obtaining module 3706 is also used to determine the lobe image plane construction model based on the lobe image section information; construct the lobe image plane section corresponding to the lobe segmentation image based on the lobe image plane construction model; or, perform layer-by-layer reconstruction based on the lobe curved section information to obtain the lobe image curved cylinder; select the intersection of the lobe image curved cylinder and the lobe segmentation image to obtain the lobe image curved section.

[0294] In one embodiment, the lobe image section obtaining module 3706 is also used to perform two-dimensional projection on the lobe curved section information to obtain a lobe curved section projection point set; generate a lobe curved section projection curve based on the lobe curved section projection point set; and perform layer-by-layer reconstruction based on the lobe curved section projection curve to obtain a lobe image curved cylinder.

[0295] In one embodiment, the lung segmentation image obtaining module 3708 is also used to segment the lung lobe segmentation image according to the plane section of the lung lobe image to obtain each unlabeled lung segmentation image; mark each unlabeled lung segmentation image according to the lung segment image position relationship of the lung lobe segmentation image to obtain each lung segmentation image; or, segment the lung lobe segmentation image according to the surface section of the lung lobe image to obtain each unlabeled lung segmentation image; traverse the lung segment image position relationship of the lung lobe segmentation image, mark each unlabeled lung segmentation image according to the traversal result, and obtain each lung segmentation image.

[0296] Each module in the above-mentioned lung segment image determination device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0297] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig.38 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store server data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for determining a lung segment image is implemented.

[0298] Those skilled in the art will understand that Fig.38 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0299] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0300] In one embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0301] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in the above-mentioned method embodiments.

[0302] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0303] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0304] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0305] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for determining a lung segment image, characterized in that: The method comprises: Segmenting the lung scan image of the target object to obtain segmented images of each bronchial segment and each lung lobe; Inputting the lung scan image into a vascular structure root node extraction model to obtain each vascular structure root node; Inputting each of the bronchial segmentation images and each of the vascular structure root nodes into a lung lobe image section reconstruction model to obtain a lung lobe image section corresponding to each of the lung lobe segmentation images; The lung lobe segmentation images corresponding to the lung lobe image sections are segmented to obtain lung segmentation images.

2. The method according to claim 1, characterized in that The step of inputting each of the bronchial segmentation images and each of the vascular structure root nodes into a lung lobe image section reconstruction model to obtain a lung lobe image section corresponding to each of the lung lobe segmentation images includes: Inputting each of the bronchial segmentation images and each of the vascular structure root nodes into a lung lobe image section reconstruction model to obtain lung lobe image section information corresponding to each of the lung lobe segmentation images; The lung lobe image sections corresponding to each of the lung lobe segmented images are reconstructed according to the lung lobe image section information corresponding to each of the lung lobe segmented images.

3. The method according to claim 2, characterized in that The step of segmenting the lung scan image of the target object to obtain segmented images of each bronchial segment includes: Performing vessel segmentation on the lung scan image to obtain a pulmonary artery segmentation image, a pulmonary vein segmentation image, and a bronchial segmentation image; The bronchial segmentation image is segmented into bronchial segments according to the pulmonary artery segmentation image and the pulmonary vein segmentation image to obtain each bronchial segmentation image.

4. The method according to claim 3, characterized in that The step of inputting each of the bronchial segmentation images and each of the vascular structure root nodes into a lung lobe image section reconstruction model to obtain lung lobe image section information corresponding to each of the lung lobe segmentation images includes: For any of the lung lobe segmentation images, selecting at least two target bronchial segmentation images from each of the bronchial segmentation images, and selecting a target vascular structure root node from each of the vascular structure root nodes; Determining a lung lobe reconstruction boundary point set from each of the target bronchial segment segmentation images using the target vascular structure root node; The lung lobe image section information is generated according to the lung lobe reconstruction boundary line point set.

5. The method according to claim 4, characterized in that The step of determining a set of pulmonary lobe reconstruction boundary points from each of the target bronchial segment segmentation images using the target vascular structure root node includes: Performing two-dimensional projection on each of the target bronchial segment segmentation images and the target vascular structure root node to obtain a bronchial segment projection image; The bronchial segment projection image is input into a boundary point set extraction model to obtain the lung lobe reconstruction boundary point set.

6. The method according to claim 5, characterized in that The pulmonary lobe reconstruction boundary line point set includes a pulmonary lobe plane boundary line point set and a pulmonary lobe curved surface boundary line point set; the bronchial segment projection image is input into the boundary line point set extraction model to obtain the pulmonary lobe reconstruction boundary line point set, including: Performing convex hull extraction on the bronchial segment projection image to obtain a bronchial segment convex hull image; generating a first plane distance field image and a second plane distance field image according to the bronchial segment convex hull image; fusing the first plane distance field image and the second plane distance field image to obtain the lung lobe plane boundary line point set; or, generating a first curved surface distance field image and a second curved surface distance field image according to the bronchial segment projection image; The first curved surface distance field image and the second curved surface distance field image are fused to obtain the lung lobe curved surface boundary line point set.

7. The method according to claim 6, characterized in that The pulmonary lobe image section information includes pulmonary lobe plane section information and pulmonary lobe curved section information; the generating of the pulmonary lobe image section information according to the pulmonary lobe reconstruction boundary point set includes: Determine two lobe plane reconstruction key points from the lobe plane boundary point set; Performing dimension conversion on each of the lung lobe plane reconstruction key points to obtain the lung lobe plane section information; or, Fitting the boundary point set of the lung lobe surface to obtain a boundary point set fitting curve; Sampling from the boundary point set fitting curve to obtain a number of lung lobe surface reconstruction key points; Dimension conversion is performed on each of the lung lobe surface reconstruction key points to obtain the lung lobe surface section information.

8. The method according to claim 7, characterized in that The lung lobe image sections include lung lobe image plane sections and lung lobe image curved sections; and reconstructing the lung lobe image sections corresponding to each of the lung lobe segmented images according to the lung lobe image section information corresponding to each of the lung lobe segmented images includes: Determine the lung lobe image plane to construct a model according to the lung lobe image section information; Constructing a lung lobe image plane section corresponding to the lung lobe segmentation image according to the lung lobe image plane construction model; or, Reconstruct layer by layer according to the lung lobe curved section information to obtain a lung lobe image curved cylindrical volume; The intersection of the lung lobe image curved cylinder and the lung lobe segmentation image is selected to obtain the lung lobe image curved section.

9. The method according to claim 8, characterized in that The step of performing layer-by-layer reconstruction according to the lung lobe curved section information to obtain a lung lobe image curved cylindrical volume includes: Performing two-dimensional projection on the pulmonary lobe curved section information to obtain a pulmonary lobe curved section projection point set; Generate a lung lobe curved surface section projection curve according to the lung lobe curved surface section projection point set; Reconstruction is performed layer by layer according to the projection curve of the lung lobe curved section to obtain the lung lobe image curved cylindrical volume.

10. The method according to claim 8, characterized in that The step of segmenting the corresponding lung lobe segmented images according to the lung lobe image sections to obtain each lung segmented image comprises: Segment the lung lobe segmentation image according to the plane section of the lung lobe image to obtain each unlabeled lung segmentation image; Marking each of the unmarked lung segmentation images according to the positional relationship between the lung segment images and the lung segmentation images to obtain each of the lung segmentation images; or, Segmenting the lung lobe segmentation image according to the lung lobe image curved surface section to obtain each of the unlabeled lung segmentation images; The positional relationship of the lung segment images of the lung lobe segmentation images is traversed, and each of the unlabeled lung segmentation images is labeled according to the traversal result to obtain each of the lung segmentation images.

11. A lung segment image determination device, characterized in that: The device comprises: A lung image segmentation module is used to segment the lung scan image of the target object to obtain segmented images of each bronchial segment and each lung lobe; A vascular structure root node obtaining module, used for inputting the lung scan image into a vascular structure root node extraction model to obtain each vascular structure root node; A lung lobe image section obtaining module, used for inputting each of the bronchial segmentation images and each of the vascular structure root nodes into a lung lobe image section reconstruction model to obtain a lung lobe image section corresponding to each of the lung lobe segmentation images; The lung segmentation image obtaining module is used to segment the corresponding lung lobe segmentation image according to each lung lobe image section to obtain each lung segmentation image.

12. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 10 are implemented.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

Images