Image segmentation method, system, electronic device and readable storage medium

Abstract

The present invention discloses an image segmentation method, system, electronic device and readable storage medium, the segmentation method comprising: obtaining an image to be identified containing an organ with a tubular structure; obtaining blood vessel data in the image to be identified; obtaining the interface between two adjacent organ segments of the organ according to the blood vessel data; and segmenting the image to be identified based on the interface. The present application starts from the medical physiological characteristics, the trachea and the veins run together, the veins run between organs containing tubular structures such as lung segments, the endpoint data of the blood vessel segment running between two adjacent organ segments is obtained through the blood vessel position data, the spatial surface is fitted according to the endpoint data as the dividing plane of the two adjacent organ segments, and the segmentation plane is determined by using the inter-segment venous running, which is more reliable and efficient and more in line with medical theoretical standards, and the segmentation method of the present application can effectively overcome the differences in organ structures of different patients and is more universal.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to an image segmentation method, system, electronic device and readable storage medium. Background Art

[0002] Lung lesion resection has evolved from anatomical lobectomy to segmentectomy, which can more accurately preserve normal lung tissue to the maximum extent and improve the postoperative survival rate of patients with poor lung function. The use of CT image segmentation to obtain lung segments is of great significance for lung function analysis, automatic evaluation of lung lesions and surgical planning. The use of medical image CT examinations to locate lung segments before surgery and the use of image navigation during surgery can provide more accurate auxiliary strategies for lung resection surgery.

[0003] In the existing technology, manual annotation combined with machine learning algorithms are often used to train historical data to obtain a segmentation model to segment lung lobes and lung segments. Manual lung segment positioning using CT images is time-consuming, and due to the differences in lung structure among different patients, the parameters obtained through training are difficult to evaluate. Therefore, a more reliable and efficient segmentation method is needed. Summary of the invention

[0004] The technical problem to be solved by the present invention is to overcome the defects of the lung segmentation method in the prior art that it is time-consuming and has low versatility and effectiveness, and to provide an image segmentation method, system, electronic device and readable storage medium.

[0005] The present invention solves the above technical problems through the following technical solutions:

[0006] A method for segmenting an image, the method comprising:

[0007] Acquire an image of an organ containing a tubular structure to be identified;

[0008] Acquiring blood vessel data in the image to be identified;

[0009] Obtaining the interface between two adjacent organ segments of the organ according to the blood vessel data;

[0010] The image to be recognized is segmented based on the interface.

[0011] Preferably, obtaining the interface between two adjacent organ segments of the organ according to the blood vessel data comprises:

[0012] Obtaining multiple blood vessel segments and endpoint data of each blood vessel segment according to the blood vessel data;

[0013] The interface is obtained according to the endpoint data.

[0014] Preferably, obtaining the interface according to the endpoint data comprises:

[0015] classifying the plurality of blood vessel segments;

[0016] Obtain target endpoint data of blood vessel segments of the same category;

[0017] The interface is obtained by fitting according to the target endpoint data.

[0018] Preferably, the classifying the plurality of blood vessel segments comprises:

[0019] The blood vessel segments running between two adjacent organ segments are classified into the same category.

[0020] Preferably, segmenting the image to be identified based on the interface includes:

[0021] Select a target interface at random;

[0022] Each voxel point in the image to be identified is input into the surface function of the target interface to determine the organ segment to which each voxel point belongs.

[0023] Preferably, the organ containing a tubular structure includes a lung, and after acquiring the image to be identified of the organ containing a tubular structure, the segmentation method further includes:

[0024] Identifying lobar fissure feature data in the image to be identified;

[0025] Segmenting the image to be identified based on the interlobar fissure feature data to obtain a plurality of lobar images;

[0026] In the step of acquiring the blood vessel data of the image to be identified, each lung lobe image is acquired separately.

[0027] Preferably, the step of acquiring blood vessel data in the image to be identified includes:

[0028] Segmenting veins from the image to be identified, and acquiring vein position data to generate the blood vessel data;

[0029] And / or, the acquiring blood vessel data in the image to be identified includes:

[0030] The blood vessel data is generated by acquiring the bifurcation point data of the blood vessel in the image to be identified.

[0031] A segmentation system for an image, the segmentation system comprising:

[0032] An image acquisition module, used for acquiring an image to be identified containing a tubular structure organ;

[0033] A blood vessel data acquisition module, used to obtain blood vessel data in the image to be identified;

[0034] An interface generation module, used for obtaining an interface between two adjacent organ segments of the organ according to the blood vessel data;

[0035] A segmentation module is used to segment the image to be identified based on the interface.

[0036] Preferably, the interface generation module includes an endpoint data acquisition unit;

[0037] The endpoint data acquisition unit is used to obtain multiple blood vessel segments and endpoint data of each blood vessel segment according to the blood vessel data;

[0038] The interface generation module is used to obtain the interface according to the endpoint data.

[0039] Preferably, the interface generation module further includes a classification unit;

[0040] The classification unit is used to classify the multiple blood vessel segments;

[0041] The interface generation module is used to obtain target endpoint data of blood vessel segments of the same category, and to obtain the interface by fitting according to the target endpoint data.

[0042] Preferably, the classification unit is used to classify blood vessel segments running between two adjacent organ segments into the same category.

[0043] Preferably, the segmentation module further comprises a selection unit and an organ segment determination unit;

[0044] The selection unit is used to arbitrarily select a target interface;

[0045] The organ segment determination unit is used to input each voxel point in the image to be identified into the surface function of the target interface to determine the organ segment to which each voxel point belongs.

[0046] Preferably, the organ containing tubular structures includes lungs, and the segmentation system further includes an interlobar fissure data recognition module and a lung lobe segmentation module;

[0047] The interlobar fissure data recognition module is used to recognize the interlobar fissure feature data in the image to be recognized;

[0048] The lung lobe segmentation module is used to segment the image to be identified based on the lung lobe fissure feature data to obtain a plurality of lung lobe images;

[0049] The blood vessel data acquisition module is used to acquire each lung lobe image separately.

[0050] Preferably, the blood vessel data acquisition module is used to segment veins from the image to be identified, and acquire vein position data to generate the blood vessel data;

[0051] and / or,

[0052] The blood vessel data acquisition module is used to acquire the bifurcation point data of the blood vessel in the image to be identified to generate the blood vessel data.

[0053] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned image segmentation method when executing the computer program.

[0054] A computer-readable storage medium stores a computer program, wherein the program, when executed by a processor, implements the steps of the above-mentioned image segmentation method.

[0055] The positive progressive effects of the present invention are as follows: the present application starts from the medical physiological characteristics, the trachea and veins run together, the veins run between organs containing tubular structures such as lung segments, the endpoint data of the blood vessel segment running between two adjacent organ segments is obtained through the blood vessel position data, and the spatial surface is fitted according to the endpoint data as the dividing plane of the two adjacent organ segments, and the segmentation plane is determined by the inter-segment vein running, which is more reliable, efficient and more in line with medical theoretical standards, and the segmentation method of the present application can effectively overcome the differences in organ structures of different patients and is more universal. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] Figure 1 This is a flowchart of the image segmentation method according to embodiment 1 of the present invention.

[0057] Figure 2 This is a flowchart of step 30 in the image segmentation method of embodiment 1 of the present invention.

[0058] Figure 3 This is a flowchart of step 32 in the image segmentation method of embodiment 1 of the present invention.

[0059] Figure 4 This is a flowchart of step 40 in the image segmentation method of embodiment 1 of the present invention.

[0060] Figure 5 This is a flow chart of a lung image segmentation method according to Embodiment 2 of the present invention.

[0061] Figure 6 Schematic diagram of the VB-Net neural network structure in the image segmentation method of Example 2 of the present invention.

[0062] Figure 7This is a schematic diagram of the lung lobe segmentation result in the image segmentation method of Example 2 of the present invention.

[0063] Figure 8a Schematic diagram of left pulmonary blood vessel segmentation in the image segmentation method of Example 2 of the present invention.

[0064] Figure 8b It is a schematic diagram of right pulmonary blood vessel segmentation in the image segmentation method of Example 2 of the present invention.

[0065] Fig. 9 It is a schematic diagram of the lung segmentation result in the image segmentation method of Example 2 of the present invention.

[0066] Fig.10 Schematic diagram of the relationship between the segmentation plane of the posterior segment of the left lung apex and the interception of the left upper lobe of the lung in the image segmentation method of Example 2 of the present invention.

[0067] Fig.11 This is a flow chart of a liver image segmentation method according to Embodiment 2 of the present invention.

[0068] Fig.12 This is a module schematic diagram of an image segmentation system according to Embodiment 3 of the present invention.

[0069] Fig.13 This is a module schematic diagram of an image segmentation system according to Embodiment 4 of the present invention.

[0070] Fig.14 This is a schematic diagram of the structure of an electronic device according to Embodiment 5 of the present invention. DETAILED DESCRIPTION

[0071] The present invention is further described below by way of examples, but the present invention is not limited to the scope of the examples.

[0072] Example 1

[0073] Figure 1 Flow chart of the image segmentation method provided in this embodiment. This embodiment involves a specific implementation process of obtaining an image to be identified containing a tubular structure organ, identifying the image to be identified to obtain blood vessel data, obtaining the interface between two adjacent organ segments of the organ based on the blood vessel data, and then segmenting the image to be identified based on the interface. Figure 1 As shown, the segmentation method includes:

[0074] Step 10, obtaining an image to be identified containing a tubular structure organ;

[0075] Specifically, organs containing tubular structures include lungs, livers, etc. The tubular structure refers to blood vessels running in the organ, and the organ can be segmented based on the blood vessels. The image to be identified can be obtained based on commonly used clinical medical imaging technology, which can specifically include: ultrasound imaging (B-ultrasound, M-ultrasound), X-ray and CT imaging, MRI imaging, PET imaging, etc.

[0076] Step 20: Obtain blood vessel data in the image to be identified;

[0077] Specifically, the above-mentioned image to be identified can be identified based on a general image recognition algorithm (including but not limited to a neural network algorithm, an artificial intelligence algorithm, etc.), blood vessels can be identified from the image to be identified, and blood vessel position data can be obtained to generate blood vessel data. Preferably, the blood vessel data can be the position data of venous blood vessels; or, blood vessel data can be further generated by obtaining the bifurcation point data of each blood vessel in the image to be identified. It should be noted that when identifying blood vessels based on a general image recognition algorithm, the image to be identified can be directly input into the image recognition algorithm for identification, or the image can be preprocessed such as image resolution adjustment, image normalization, etc., and then the preprocessed image can be input into the image recognition algorithm for identification. The blood vessel data involved above can specifically be the position data of the blood vessels after the three-dimensional coordinate system of the organ is constructed.

[0078] Step 30, obtaining the interface between two adjacent organ segments of the organ according to the blood vessel data;

[0079] Specifically, the organ segment is used to characterize an independent unit after the organ is segmented. Taking the organ lung as an example, the lung segment generally refers to the bronchial lung segment in medicine, which refers to an independent blood oxygen exchange unit. The interface can be the interlobar segmentation surface of the lung or the intralobar segmentation surface. The interlobar segmentation surface is a part of the interlobar fissure, which is used to divide different lobes of the lung. The intralobar segmentation surface refers to the segmentation surface after further division based on the lobes of the lung. Similarly, taking the organ liver as an example, the interface of the liver can be the interlobar segmentation surface of the liver or the intralobar segmentation surface. The interlobar segmentation surface is used to divide different lobes of the liver. The intralobar segmentation surface refers to the segmentation surface after further division based on the lobes of the liver. In this embodiment, after obtaining the aforementioned vascular position data, the vascular position data running between two adjacent organ segments is obtained, and a spatial surface is fitted according to the vascular position data as the interface between the two adjacent organ segments.

[0080] Step 40: Segment the image to be recognized based on the interface.

[0081] Specifically, after the interface is determined, the organ can be divided into different segments according to the interface. Please refer to FIG8 , which takes the lung as an example and shows a schematic diagram of the lung after segmentation.

[0082] The above-mentioned image segmentation method is used to segment the image to be identified containing a tubular structure organ, obtain the interface between two adjacent organ segments of the organ through the blood vessel data in the image to be identified, and segment the image to be identified based on the interface. The present application uses blood vessel data to determine the segmentation plane, which is more reliable and efficient and more in line with medical theory standards. In addition, the segmentation method of the present application can effectively overcome the differences in organ structures of different patients and is more universal.

[0083] Figure 2 The specific implementation of step 30 in the image segmentation method of this embodiment is shown, which specifically includes:

[0084] Step 31, obtaining multiple blood vessel segments and endpoint data of each blood vessel segment according to the blood vessel data;

[0085] Specifically, endpoint data refers to the endpoint position data of each segment of blood vessels. The segments are named according to their relative positions in the organs and the segment angles. The starting point and ending point of each segment are used as the endpoints of the segment. The endpoint data constitute the above-mentioned blood vessel data. Specifically, blood vessels continuously branch during their course in the organs. When the image to be identified is identified, the bifurcation point data of each blood vessel in the image to be identified is obtained. Taking the organ lung as an example, the running blood vessels include lobar blood vessels, segmental blood vessels and subsegmental blood vessels. For lobar blood vessels, the starting point is the starting point of each blood vessel, and the ending point is the bifurcation point where the lobar blood vessels bifurcate into the segmental blood vessels. For the segmental blood vessels, the starting point is the ending point of the lobar blood vessels, and the ending point is the bifurcation point where the segmental blood vessels bifurcate into the subsegmental blood vessels. For the subsegmental blood vessels, the starting point is the ending point of the segmental blood vessels, and the ending point is the ending point where the segmental blood vessels bifurcate into the subsegmental blood vessels.

[0086] Step 32: Obtain the interface according to the endpoint data.

[0087] Specifically, after acquiring the endpoint data, the above-mentioned space curved surface is fitted according to the position data of the endpoint as the interface between the two adjacent organ segments.

[0088] In the above-mentioned image segmentation method, in order to comply with medical theory standards, the segmentation plane of the organ is determined by the blood vessel segments running in the organ. Specifically, the blood vessel segments are acquired using the relative information of the anatomical space, and the interface is further determined based on the endpoint data of the blood vessel segments. The segmentation method is faster and more efficient.

[0089] Among them, see Figure 3 , Figure 3 The specific implementation of step 32 in the image segmentation method of this embodiment is shown, which specifically includes:

[0090] Step 321, classifying multiple blood vessel segments;

[0091] Specifically, by classifying the blood vessel segments running between two adjacent organ segments into the same category, the endpoints of the blood vessel segments of the same category, that is, the inter-segment blood vessel segments between adjacent organ segments, are further obtained as fitting points of subsequent organ segment surfaces.

[0092] Step 322, obtaining target endpoint data of blood vessel segments of the same category;

[0093] The target endpoint data refers to the endpoint position data of the same category of blood vessel segments. When obtaining the interface, each category of blood vessel segments needs to be processed separately.

[0094] Step 323: Obtain an interface by fitting according to the target endpoint data.

[0095] Specifically, after constructing the three-dimensional coordinate system of the organ, the position data of the target endpoint is obtained, and a surface is fitted according to the position data of the target endpoint as the interface. The least squares method can be used for fitting, but it is not limited to this method, and other fitting algorithms can also be used for calculation. A surface function is obtained by fitting, and the spatial surface represented by the surface function is used as the interface.

[0096] In the above-mentioned image segmentation method, the relative information of anatomical space is used to determine the segmentation plane of the organ with the blood vessel segments running between two adjacent organ segments, which is more in line with medical theoretical standards. At the same time, in this embodiment, the segmentation method is faster and more efficient based on the acquired position data. At the same time, it is more universal for differentiating different data.

[0097] Figure 4 The specific implementation of step 40 in the image segmentation method of this embodiment is shown, which specifically includes:

[0098] Step 41, arbitrarily select a target interface;

[0099] Specifically, the target interface refers to an arbitrarily selected interface.

[0100] Step 42: Input each voxel point in the image to be identified into the surface function of the target interface to determine the organ segment to which each voxel point belongs.

[0101] Specifically, after constructing the three-dimensional coordinate system of the organ, the surface function uses the aforementioned three-dimensional coordinate system as a reference, confirms each voxel point in the image to be identified containing the tubular structure organ one by one, substitutes the coordinate point of each voxel point into the surface function expression of the target interface, and calculates the relative relationship between each voxel point and the segmented surface to determine which side of the target interface each voxel point is on. The two sides are the two sides in any direction in the space separated by the plane, which can be the upper side or the lower side or the left side or the right side or the front side or the back side. It should be noted that the above operations need to be performed on all interfaces separately to determine the organ segment to which each voxel point belongs. For example, starting from the reference of the three-dimensional coordinate system, two interfaces are obtained in the segmentation of a certain organ, which are interface 1 and interface 2 from top to bottom. The organ is divided into segment 1, segment 2 and segment 3. After the coordinate point of a voxel point is respectively brought into the surface function corresponding to interface 1 and interface 2, the voxel point is on the lower side of interface 1 and on the upper side of interface 2, and the organ segment segment 2 to which the voxel point belongs can be obtained. For all voxel points and all interfaces, according to the surface function of each interface in the segmented results, the organ segments to which the remaining voxel points belong are marked until all segmentations are completed.

[0102] In this embodiment, the organ containing tubular structure can be lung, liver, etc. Taking lung as an example, the bronchus in the lung gradually becomes thinner from the hilum to the subpleural area. In terms of physiological characteristics, the pulmonary blood vessels accompany the trachea and appear as highlights on thin-layer CT (layer thickness <= 1.25 mm) and as obvious blood vessel-poor bands on thick-layer CT. The neural network algorithm can be used to independently segment the vascular data on the chest enhancement data, especially there are obvious inter-segment vascular branches running between each lung segment. The endpoint data of the vascular segment running between two adjacent organ segments is obtained through the vascular position data, and the spatial surface is fitted according to the endpoint data as the division plane of the two adjacent organ segments. In this embodiment, the segmentation plane is determined by the inter-segment vein running, which is more reliable and efficient and more in line with medical theoretical standards. In addition, the segmentation method of the present application can effectively overcome the differences in organ structures of different patients and is more universal.

[0103] Example 2

[0104] This embodiment is a further improvement on the basis of embodiment 1. Figure 5 , showing a segmentation method of a lung image, after step 10, the segmentation method further comprises:

[0105] Step 11, identifying the interlobar fissure feature data in the image to be identified;

[0106] Specifically, the lung lobe is an independent subunit of the lung's physiological function. Each lobe is independently surrounded by the pleura, appears as a highlight on thin-layer CT (layer thickness <= 1.25 mm), and appears as an obvious avascular zone on thick-layer CT. The neural network model is used to identify the interlobar fissure characteristics.

[0107] Step 12: segment the image to be identified based on the interlobar fissure feature data to obtain multiple lobe images;

[0108] Specifically, a neural network model is used to segment different lung lobes according to the interlobar fissure features. When segmenting the lung segments, a deep learning method is first used to segment the lung lobes. Any convolutional neural network can be used for the segmentation of lung lobes and pulmonary blood vessels in this application.

[0109] Step 20 further specifically includes:

[0110] Step 201: Obtain blood vessel data in each lung lobe image in the image to be identified.

[0111] In this embodiment, taking the lung as an example, in order to reduce the segmentation time and improve the efficiency of lung segmentation, lung lobe segmentation is first performed based on a deep learning method, and then each lung lobe is segmented separately.

[0112] Taking lung segmentation as an example, the solution of this embodiment is further explained with a specific example:

[0113] (1) Obtaining an image of an organ containing a tubular structure to be identified

[0114] The chest enhanced CT data is obtained and the image is preprocessed, specifically including: resampling to adjust the image resolution and performing image normalization. The present invention uses the lung window: [-600, 750] as the mean and standard deviation to adjust the image. The formula is:

[0115]

[0116] Where μ is the mean, σ is the standard deviation, I is the image resolution, and I′ is the adjusted image resolution.

[0117] (2) Obtaining blood vessel data in the image to be identified

[0118] (a) Using a neural network method to segment pulmonary lobes It should be noted that any convolutional neural network can be used for segmenting pulmonary lobes and pulmonary veins in this application.

[0119] VB-Net neural network can be used, the network structure is as follows Figure 6 As shown in Figure 7As shown (upper lobe, middle lobe, lower lobe). The segmentation process is completed by two networks with different resolutions. The low-resolution network adopts [5mm, 5mm, 5mm] voxel resolution, which is mainly used for coarse positioning of the lung lobes and can effectively remove background interference information. Then the original image filtered after coarse positioning of the lung lobes is input into the high-resolution network. The high-resolution network adopts [1mm, 1mm, 1mm] voxel resolution, which can segment fine lung lobe boundaries. Among them, due to the large dimension of 3D images, the training adopts block training. The block size can be selected according to the physical video memory. The optional range is: 64~256, preferably 96*96*96. The optimization function selects Dice Loss (set similarity measurement function), and the training optimizer selects Adam (a first-order optimization algorithm that can replace the traditional stochastic gradient descent process).

[0120] (b) Pulmonary vein segmentation using neural network method. Pulmonary vein segmentation is similar to lung lobe segmentation, and the segmentation process from low-resolution network to high-resolution network is also used. The diameter of the pulmonary vein terminal is generally less than 1mm, so the high-resolution network adopts [0.5mm, 0.5mm, 0.5mm]. The training process, optimization function and optimizer selection are the same as those in lung lobe segmentation.

[0121] (c) Extract the pulmonary vein centerline and use the bifurcation points on the centerline (i.e., points with neighborhood > 2) to segment the blood vessels, as follows: Figure 8a and Figure 8b As shown, the segments are named according to the lung lobes where the segments are located, their relative positions, and the segment angles (see Table 1), and the starting point and the ending point of each segment are used as the endpoints of the segment, and the endpoint data constitute the blood vessel data.

[0122] Table 1 Lung segment nomenclature

[0123]

[0124]

[0125] (3) Obtaining the interface between two adjacent organ segments of the organ based on the blood vessel data

[0126] According to the description of the left and right pulmonary veins in Tables 2 and 3 below, the vascular segments running between two adjacent organ segments are divided into the same category, and the endpoints of the vascular segments of the same category, that is, the intersegmental veins between adjacent pulmonary segments, are obtained as the fitting points of the pulmonary segment surface. The least squares method is used for fitting (including but not limited to this method, and other fitting algorithms can also be used for calculation). In order to avoid excessive changes in curvature and ensure the smoothness of the surface, the present application uses a cubic surface for fitting. Any cubic surface can be expressed by a power basis function as follows:

[0127] z=a1x 3 +a2y3 +a3x 2 y+a4xy 2 +a5x 2 +a6y 2 +a7xy+a8x+a9y+a 10

[0128] Among them, a1…a 10 is the function coefficient, x, y, z are the three-dimensional coordinate values ​​respectively.

[0129] Table 2 Course of left pulmonary vein branches

[0130]

[0131]

[0132] Table 3 Course of right pulmonary vein branches

[0133]

[0134] (4) Segmenting the image to be identified based on the interface

[0135] The surface fitting equation is established based on the anatomical coordinate system. The segmentation results of all lung segments are as follows: Fig. 9 As shown in the figure, the right side of the human body (Right, R direction) is the x direction, from back to front (Anterior, A direction) is the y direction, and from foot to head (Superior, S direction) is the z direction. Substitute the coordinate points of the lung lobe into the function expression of the fitting segmented plane to calculate the relative relationship between each voxel point of the lung lobe and the segmented surface.

[0136] Calculate a1x 3 +a2y 3 +a3x 2 y+a4xy 2 +a5x 2 +a6y 2 +a7xy+a8x+a9y+a 10 -z, if the calculation result is >= 0, the voxel point is located on the upper side of the segmentation surface and is marked as a segment; if the calculation result is < 0, the voxel point is located on the lower side of the segmentation surface and is marked as another segment. According to the segmented results, mark the remaining adjacent lung segments until all lung segments are segmented. See Fig.10 , showing the relationship between the segmental plane of the posterior segment of the left lung apex and the interception of the left upper lobe.

[0137] In this example, the organ containing tubular structure takes the lung as an example. The bronchus in the lung gradually becomes thinner from the hilum to the subpleural area. In terms of physiological characteristics, the pulmonary blood vessels accompany the trachea and appear as highlights on thin-layer CT (layer thickness <= 1.25 mm) and as obvious blood vessel-poor bands on thick-layer CT. The neural network algorithm can be used to independently segment the vascular data on the chest enhancement data, especially there are obvious inter-segment vascular branches running between the lung segments. The endpoint data of the vascular segment running between two adjacent lung segments is obtained through the vascular position data, and the spatial surface is fitted according to the endpoint data as the dividing plane of the two adjacent lung segments. This embodiment uses the inter-segment veins to determine the segmentation plane, which is more reliable and efficient and more in line with medical theoretical standards. In addition, the segmentation method of the present application can effectively overcome the differences in lung structure among different patients and is more universal.

[0138] In this embodiment, the organ containing the tubular structure may also include the liver, see Fig.11 , showing a segmentation method of a liver image. For the segmentation of the liver image, after step 10, the segmentation method further includes:

[0139] Step 13, identifying the image to be identified to obtain hepatic vein feature data;

[0140] Specifically, when segmenting the liver, the neural network model is used to obtain the hepatic vein feature data.

[0141] Step 14: segment the image to be identified based on the hepatic vein feature data to obtain multiple liver lobe images;

[0142] Specifically, after obtaining the characteristic data of the hepatic vein, in actual applications, the neural network model is used to obtain the liver's ligamentous fissures (round ligament fissure, venous ligament fissure), gallbladder fossa, etc., and different liver lobes are segmented based on the above data. Therefore, when segmenting the liver segments, the deep learning method is first used to segment the liver lobes. Any convolutional neural network can be used for the segmentation of liver lobes and hepatic veins in this application.

[0143] Step 20 further specifically includes:

[0144] Step 202: Obtain blood vessel data in each liver lobe image in the image to be identified.

[0145] In this embodiment, taking the liver as an example, in order to reduce the segmentation time and improve the efficiency of liver segmentation, liver lobe segmentation is first performed based on a deep learning method, and then each liver lobe is segmented separately.

[0146] Example 3

[0147] A segmentation system for an image, such as Fig.12 As shown, the segmentation system includes:

[0148] An image acquisition module 1 is used to acquire an image to be identified of an organ containing a tubular structure;

[0149] Specifically, organs containing tubular structures include lungs, livers, etc. The tubular structure refers to blood vessels running in the organ, and the organ can be segmented based on the blood vessels. The image to be identified can be obtained based on commonly used clinical medical imaging technology, which can specifically include: ultrasound imaging (B-ultrasound, M-ultrasound), X-ray and CT imaging, MRI imaging, PET imaging, etc.

[0150] Step 20: Obtain blood vessel data in the image to be identified;

[0151] A blood vessel data acquisition module 2, used to obtain blood vessel data in the image to be identified;

[0152] Specifically, the above-mentioned image to be identified can be identified based on a general image recognition algorithm (including but not limited to a neural network algorithm, an artificial intelligence algorithm, etc.), blood vessels can be identified from the image to be identified, and blood vessel position data can be obtained to generate blood vessel data; or, blood vessel data can be further generated by obtaining bifurcation point data of each blood vessel in the image to be identified. It should be noted that when identifying blood vessels based on a general image recognition algorithm, the image to be identified can be directly input into the image recognition algorithm for identification, or the image can be preprocessed such as image resolution adjustment, image normalization, etc., and then the preprocessed image can be input into the image recognition algorithm for identification. The blood vessel data involved above can specifically be the position data of the blood vessels after the three-dimensional coordinate system of the organ is constructed.

[0153] An interface generation module 3, used for obtaining an interface between two adjacent organ segments of the organ according to the blood vessel data;

[0154] Specifically, the organ segment is used to characterize an independent unit after the organ is segmented. Taking the organ lung as an example, the lung segment generally refers to the bronchial lung segment in medicine, which refers to an independent blood oxygen exchange unit. The interface can be the interlobar segmentation surface of the lung or the intralobar segmentation surface. The interlobar segmentation surface is a part of the interlobar fissure, which is used to divide different lobes of the lung. The intralobar segmentation surface refers to the segmentation surface after further division based on the lobes of the lung. Similarly, taking the organ liver as an example, the interface of the liver can be the interlobar segmentation surface of the liver or the intralobar segmentation surface. The interlobar segmentation surface is used to divide different lobes of the liver. The intralobar segmentation surface refers to the segmentation surface after further division based on the lobes of the liver. In this embodiment, after obtaining the aforementioned vascular position data, the vascular position data running between two adjacent organ segments is obtained, and a spatial surface is fitted according to the vascular position data as the interface between the two adjacent organ segments.

[0155] The segmentation module 4 is used to segment the image to be identified based on the interface.

[0156] Specifically, after the interface is determined, the organ can be divided into different segments according to the interface.

[0157] The above-mentioned image segmentation system is used to segment the image to be identified containing a tubular structure organ, obtain the interface between two adjacent organ segments of the organ through the blood vessel data in the image to be identified, and segment the image to be identified based on the interface. The present application uses blood vessel data to determine the segmentation plane, which is more reliable and efficient and more in line with medical theory standards. In addition, the segmentation method of the present application can effectively overcome the differences in organ structures of different patients and is more universal.

[0158] Wherein, the interface generation module 3 includes an endpoint data acquisition unit 301;

[0159] The endpoint data acquisition unit 301 is used to obtain multiple blood vessel segments and endpoint data of each blood vessel segment according to the blood vessel data;

[0160] Specifically, endpoint data refers to the endpoint position data of each segment of blood vessels. The segments are named according to their relative positions in the organs and the segment angles. The starting point and ending point of each segment are used as the endpoints of the segment. The endpoint data constitute the above-mentioned blood vessel data. Specifically, blood vessels continuously branch during their course in the organs. When the image to be identified is identified, the bifurcation point data of each blood vessel in the image to be identified is obtained. Taking the organ lung as an example, the running blood vessels include lobar blood vessels, segmental blood vessels and subsegmental blood vessels. For lobar blood vessels, the starting point is the starting point of each blood vessel, and the ending point is the bifurcation point where the lobar blood vessels bifurcate into the segmental blood vessels. For the segmental blood vessels, the starting point is the ending point of the lobar blood vessels, and the ending point is the bifurcation point where the segmental blood vessels bifurcate into the subsegmental blood vessels. For the subsegmental blood vessels, the starting point is the ending point of the segmental blood vessels, and the ending point is the ending point where the segmental blood vessels bifurcate into the subsegmental blood vessels.

[0161] The interface generation module 3 is used to obtain the interface according to the endpoint data.

[0162] Specifically, after acquiring the endpoint data, the above-mentioned space curved surface is fitted according to the position data of the endpoint as the interface between the two adjacent organ segments.

[0163] In the above-mentioned image segmentation method, in order to comply with medical theory standards, the segmentation plane of the organ is determined by the blood vessel segments running in the organ. Specifically, the blood vessel segments are acquired using the relative information of the anatomical space, and the interface is further determined based on the endpoint data of the blood vessel segments. The segmentation method is faster and more efficient.

[0164] Furthermore, the interface generation module 3 also includes a classification unit 302;

[0165] The classification unit 302 is used to classify the multiple blood vessel segments;

[0166] Specifically, by classifying the blood vessel segments running between two adjacent organ segments into the same category, the endpoints of the blood vessel segments of the same category, that is, the inter-segment blood vessel segments between adjacent organ segments, are further obtained as fitting points of subsequent organ segment surfaces.

[0167] The interface generation module 3 is used to obtain target endpoint data of blood vessel segments of the same category, and obtain the interface by fitting according to the target endpoint data.

[0168] The target endpoint data refers to the endpoint position data of the same category of blood vessel segments. When obtaining the interface, each category of blood vessel segments needs to be processed separately. After constructing the three-dimensional coordinate system of the organ, the position data of the target endpoint is obtained, and a surface is fitted as the interface according to the position data of the target endpoint. The least squares method can be used for fitting, but it is not limited to this method. Other fitting algorithms can also be used for calculation. A surface function is obtained by fitting, and the spatial surface embodied by the surface function is used as the interface.

[0169] The above-mentioned image segmentation system utilizes the relative information of anatomical space to determine the segmentation plane of the organ based on the blood vessel segments running between two adjacent organ segments, which is more in line with medical theoretical standards. At the same time, in this embodiment, the segmentation method is faster and more efficient based on the acquired position data. At the same time, it is also more universal for differentiating different data.

[0170] In this embodiment, the segmentation module 4 further includes a selection unit 401 and an organ segment determination unit 402;

[0171] The selection unit 401 is used to arbitrarily select a target interface;

[0172] Specifically, the target interface refers to an arbitrarily selected interface.

[0173] The organ segment determination unit 402 is used to input each voxel point in the image to be identified into the surface function of the target interface to determine the organ segment to which each voxel point belongs.

[0174] Specifically, after constructing the three-dimensional coordinate system of the organ, the surface function uses the aforementioned three-dimensional coordinate system as a reference, confirms each voxel point in the image to be identified containing the tubular structure organ one by one, substitutes the coordinate point of each voxel point into the surface function expression of the target interface, and calculates the relative relationship between each voxel point and the segmented surface to determine which side of the target interface each voxel point is on. The two sides are the two sides in any direction in the space separated by the plane, which can be the upper side or the lower side or the left side or the right side or the front side or the back side. It should be noted that the above operations need to be performed on all interfaces separately to determine the organ segment to which each voxel point belongs. For example, starting from the reference of the three-dimensional coordinate system, two interfaces are obtained in the segmentation of a certain organ, which are interface 1 and interface 2 from top to bottom. The organ is divided into segment 1, segment 2 and segment 3. After the coordinate point of a voxel point is respectively brought into the surface function corresponding to interface 1 and interface 2, the voxel point is on the lower side of interface 1 and on the upper side of interface 2, and the organ segment segment 2 to which the voxel point belongs can be obtained. For all voxel points and all interfaces, according to the surface function of each interface in the segmented results, the organ segments to which the remaining voxel points belong are marked until all segmentations are completed.

[0175] In this embodiment, the organ containing tubular structure can be lung, liver, etc. Taking lung as an example, the bronchus in the lung gradually becomes thinner from the hilum to the subpleural area. In terms of physiological characteristics, the pulmonary blood vessels accompany the trachea and appear as highlights on thin-layer CT (layer thickness <= 1.25 mm) and as obvious blood vessel-poor bands on thick-layer CT. The neural network algorithm can be used to independently segment the vascular data on the chest enhancement data, especially there are obvious inter-segment vascular branches running between each lung segment. The endpoint data of the vascular segment running between two adjacent organ segments is obtained through the vascular position data, and the spatial surface is fitted according to the endpoint data as the division plane of the two adjacent organ segments. In this embodiment, the segmentation plane is determined by the inter-segment vein running, which is more reliable and efficient and more in line with medical theoretical standards. In addition, the segmentation method of the present application can effectively overcome the differences in organ structures of different patients and is more universal.

[0176] Example 4

[0177] This embodiment is a further improvement on the basis of embodiment 3, wherein the organ containing the tubular structure includes the lung, such as Fig.13 As shown, the segmentation system further includes an interlobar fissure data identification module 5 and a lung lobe segmentation module 6;

[0178] The interlobar fissure data recognition module 5 is used to recognize the interlobar fissure feature data in the image to be recognized;

[0179] Specifically, the lung lobe is an independent subunit of the lung's physiological function. Each lobe is independently surrounded by the pleura, and appears as a highlight on thin-layer CT (layer thickness <= 1.25 mm) and as an obvious interlobar fissure with avascular bands on thick-layer CT. The neural network model is used to obtain the interlobar fissure characteristics.

[0180] The lung lobe segmentation module 6 is used to segment the image to be identified based on the lung lobe fissure feature data to obtain multiple lung lobe images;

[0181] Specifically, a neural network model is used to segment different lung lobes according to the interlobar fissure features. Therefore, when segmenting the lung segments, a deep learning method is first used to segment the lung lobes. Any convolutional neural network can be used for the segmentation of lung lobes and pulmonary blood vessels in this application.

[0182] The blood vessel data acquisition module 2 is used to acquire the image of each lung lobe respectively.

[0183] In this embodiment, taking the lung as an example, in order to reduce the segmentation time and improve the efficiency of lung segmentation, lung lobe segmentation is first performed based on a deep learning method, and then each lung lobe is segmented separately.

[0184] In this embodiment, the organ containing the tubular structure may also include the liver, see Fig.11 , the segmentation system also includes a hepatic vein data identification module 7 and a liver lobe segmentation module 8;

[0185] The hepatic vein data recognition module 7 is used to recognize the image to be recognized to obtain hepatic vein feature data;

[0186] Specifically, when segmenting the liver, the neural network model is used to obtain the hepatic vein feature data.

[0187] The liver lobe segmentation module 8 is used to segment the image to be identified based on the hepatic vein feature data to obtain a plurality of liver lobe images;

[0188] Specifically, after obtaining the characteristic data of the hepatic vein, in actual applications, the neural network model is used to obtain the liver's ligamentous fissures (round ligament fissure, venous ligament fissure), gallbladder fossa, etc., and different liver lobes are segmented based on the above data. Therefore, when segmenting the liver segments, the deep learning method is first used to segment the liver lobes. Any convolutional neural network can be used for the segmentation of liver lobes and hepatic veins in this application.

[0189] The blood vessel data acquisition module 2 is used to identify each liver lobe image separately.

[0190] In this embodiment, taking the liver as an example, in order to reduce the segmentation time and improve the efficiency of liver segmentation, liver lobe segmentation is first performed based on a deep learning method, and then each liver lobe is segmented separately.

[0191] Example 5

[0192] An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the image segmentation method described in Embodiment 1 or 2 when executing the computer program.

[0193] Fig.14 A schematic diagram of the structure of an electronic device provided in this embodiment. Fig.14 A block diagram of an exemplary electronic device 90 suitable for use in implementing embodiments of the present invention is shown. Fig.14 The electronic device 90 shown is only an example and should not limit the functions and scope of use of the embodiments of the present invention. Fig.12 As shown, the electronic device 90 may be in the form of a general-purpose computing device, for example, it may be a server device. The components of the electronic device 90 may include, but are not limited to: at least one processor 91, at least one memory 92, and a bus 93 connecting different system components (including the memory 92 and the processor 91).

[0194] The bus 93 includes a data bus, an address bus, and a control bus.

[0195] The memory 92 may include a volatile memory, such as a random access memory (RAM) 921 and / or a cache memory 922 , and may further include a read-only memory (ROM) 923 .

[0196] The memory 92 may also include a program tool 925 having a set (at least one) of program modules 924, such program modules 924 including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include the implementation of a network environment.

[0197] The processor 91 executes various functional applications and data processing by running computer programs stored in the memory 92 .

[0198] The electronic device 90 may also communicate with one or more external devices 94 (e.g., keyboards, pointing devices, etc.). Such communication may be performed via an input / output (I / O) interface 95. Furthermore, the electronic device 90 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 96. The network adapter 96 communicates with other modules of the electronic device 90 via a bus 93. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 90, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems, etc.

[0199] It should be noted that although several units / modules or sub-units / modules of the electronic device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided into multiple units / modules to be embodied.

[0200] Example 6

[0201] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the image segmentation method described in embodiment 1 or 2.

[0202] The readable storage medium may include but is not limited to: a portable disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device or any suitable combination of the above.

[0203] In a possible implementation, the present invention can also be implemented in the form of a program product, which includes program code. When the program product runs on a terminal device, the program code is used to enable the terminal device to execute the steps of the image segmentation method described in Example 1 or 2.

[0204] Although the specific embodiments of the present invention are described above, it should be understood by those skilled in the art that this is only for illustration and the protection scope of the present invention is defined by the appended claims. Those skilled in the art may make various changes or modifications to these embodiments without departing from the principles and essence of the present invention, but these changes and modifications all fall within the protection scope of the present invention.

Claims

1. A method for segmenting an image, characterized in that: The segmentation method includes: Acquire an image of an organ to be identified containing a tubular structure; Acquiring blood vessel data in the image to be identified; Wherein, the blood vessel data includes the course of the venous vessels between two adjacent organ segments of the organ; Obtaining the interface between two adjacent organ segments of the organ according to the blood vessel data includes: Obtaining multiple blood vessel segments and endpoint data of each blood vessel segment according to the course of the venous blood vessel; Obtaining the interface according to the endpoint data; The image to be recognized is segmented based on the interface.

2. The image segmentation method according to claim 1, characterized in that: The obtaining of the interface according to the endpoint data comprises: classifying the plurality of blood vessel segments; Obtain target endpoint data of blood vessel segments of the same category; The interface is obtained by fitting according to the target endpoint data.

3. The image segmentation method according to claim 2, characterized in that: The classifying the plurality of blood vessel segments comprises: The blood vessel segments running between two adjacent organ segments are classified into the same category.

4. The image segmentation method according to claim 1, characterized in that: The segmenting the image to be identified based on the interface comprises: Select a target interface at random; Each voxel point in the image to be identified is input into the surface function of the target interface to determine the organ segment to which each voxel point belongs.

5. The image segmentation method according to claim 1, characterized in that: The organ containing a tubular structure includes a lung. After acquiring the image to be identified of the organ containing a tubular structure, the segmentation method further includes: Identifying lobar fissure feature data in the image to be identified; Segmenting the image to be identified based on the interlobar fissure feature data to obtain a plurality of lobar images; In the step of acquiring the blood vessel data of the image to be identified, each lung lobe image is acquired separately.

6. The image segmentation method according to claim 1, characterized in that: The acquiring of blood vessel data in the image to be identified comprises: Segmenting veins from the image to be identified, and acquiring vein position data to generate the blood vessel data; And / or, the acquiring blood vessel data in the image to be identified includes: The blood vessel data is generated by acquiring the bifurcation point data of the blood vessel in the image to be identified.

7. A system for segmenting an image, characterized in that: The segmentation system comprises: An image acquisition module, used for acquiring an image to be identified containing a tubular structure organ; A blood vessel data acquisition module, used to obtain blood vessel data in the image to be identified; wherein the blood vessel data includes the course of the venous vessels between two adjacent organ segments of the organ; The interface generation module is used to obtain the interface between two adjacent organ segments of the organ according to the blood vessel data, and includes an endpoint data acquisition unit: An endpoint data acquisition unit, used for obtaining multiple blood vessel segments and endpoint data of each blood vessel segment according to the course of the venous blood vessel; The interface generation module is also used to obtain the interface according to the endpoint data; A segmentation module is used to segment the image to be identified based on the interface.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the image segmentation method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the image segmentation method according to any one of claims 1 to 6 are implemented.

Images