Automatic Lung Lobation Method, Device and Electronic Equipment Based on Medical Images

Abstract

The present invention provides an automatic lung lobation method based on medical images, which includes the following steps: using bronchial data to obtain regions of interest of each interlobar fissure from the lung medical image to be lobated; using the Hessian matrix to enhance each of the regions of interest to obtain corresponding enhanced images; extracting preliminary interlobar fissure data from each of the enhanced images; using an interpolation algorithm to reconstruct the preliminary interlobar fissure data to generate complete interlobar fissure data, and then performing lung lobation according to the complete interlobar fissure data. The present invention also provides an automatic lung lobation device and an electronic device based on medical images. Thereby, the present invention can achieve full-automatic lung lobation, and the lobation result is more accurate.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular, to an automatic lung lobation method, device and electronic device based on medical images. Background Art

[0002] Lung cancer has become the number one cancer killer threatening national health. Surgical operations play a decisive role in the treatment of resectable lung cancer, and minimally invasive thoracoscopic surgery improves the postoperative quality of life and survival rate of patients. For lung lesions confined to a single lung lobe while the rest of the lungs are normal, lobectomy can be performed. Lobectomy is the most commonly used surgical procedure for treating pulmonary surgical diseases at present. The loss of lung function after lobectomy is less, and it has little impact on the normal life of patients. It can reduce the loss of lung function while ensuring the resection range, achieving a good treatment effect.

[0003] Each lung tissue distributed by a pulmonary segment bronchus and its branches is called a bronchopulmonary segment. Within the pulmonary segment, the branches of the pulmonary artery run parallel to the pulmonary segment bronchus. The spatial orientation and mutual position of the bronchus in three-dimensional space determine the positions of each pulmonary segment. Obtaining a clear and accurate lung lobation structure before lobectomy has a crucial impact on the success of the operation. The prior art often adopts the method of manual lung lobation, specifically including: applying the knowledge of pulmonary anatomy, comprehensively identifying the three sets of pipelines of the hilum of the lung, intrapulmonary bronchi, pulmonary artery, and pulmonary vein on each layer of continuous lung CT images, and drawing a simple image diagram for graphic remodeling, so as to observe the distribution of the three sets of pipelines in three dimensions. It can be used to proofread the correctness of the structures observed in the images, and at the same time for further lobation and segmentation on CT images. At the same time, the spatial orientation and mutual position relationship of the three sets of pipelines of the lung in three dimensions are applied to CT images, and combined with the identification of the interlobar fissures of the lungs, to achieve lung lobation on CT images. This method of manual lung lobation is time-consuming and laborious.

[0004] There is also a prior art method for automatic and semi-automatic lung lobation and segmentation based on bronchi. According to the Thiessen polygon theory algorithm, the lobation of the bronchi corresponding to the lungs is calculated, but this method does not utilize the information in CT images, and the obtained lung lobe results often have a certain gap from the lung lobation results based on the interlobar fissures in CT images.

[0005] In summary, it is obvious that the prior art has inconveniences and defects in actual use, so it is necessary to improve. Summary of the Invention

[0006] Aiming at the above defects, the purpose of the present invention is to provide an automatic lung lobation method, device and electronic device based on medical images, which can achieve full-automatic lung lobation and the lobation results are more accurate.

[0007] To solve the above technical problems, the present invention is implemented as follows:

[0008] In a first aspect, an embodiment of the present invention provides an automatic lung lobation method based on medical images, including the following steps:

[0009] Using bronchial data, obtain the regions of interest of the interlobar fissures from the lung medical images to be lobated;

[0010] Using the Hessian matrix, enhance each of the regions of interest to obtain corresponding enhanced images;

[0011] Extract preliminary interlobar fissure data from each of the enhanced images;

[0012] Use the interpolation algorithm to reconstruct the preliminary interlobar fissure data into complete interlobar fissure data, and then perform lung lobation according to the complete interlobar fissure data.

[0013] According to the method of the present invention, the step of using bronchial data to obtain the regions of interest of the interlobar fissures from the lung medical images to be lobated includes:

[0014] Calculate the Euler characteristic values of each discrete point of the bronchus to obtain the bronchial centerline point set;

[0015] Construct a bronchial tree structure according to the bronchial centerline point set;

[0016] Automatically mark the bronchial centerlines corresponding to the five lung lobes according to the bronchial tree structure and bronchial physiological characteristics;

[0017] Calculate the distance between each pixel point in the lung and the five bronchial centerlines, and mark the pixel point with the same mark as the nearest bronchial centerline;

[0018] Obtain the set of joint points between two different marks, calculate the regression plane using the set of joint points, and expand the regression plane by a predetermined first length as the region of interest for calculating the interlobar fissure; repeat this operation until the regions of interest of each interlobar fissure are calculated, and subsequent operations are performed on each region of interest of the interlobar fissure respectively.

[0019] According to the method of the present invention, the first length is 25 - 35 mm.

[0020] According to the method of the present invention, the step of using the Hessian matrix to enhance each of the regions of interest to obtain corresponding enhanced images includes:

[0021] Modify the pixel values of the region where the pixel values in the region of interest are less than the first pixel threshold to the first pixel threshold, and modify the pixel values of the region where the pixel values are greater than the second pixel threshold to the second pixel threshold;

[0022] Construct a Gaussian filter G σ , where σ is the variance of the Gaussian filter G σ of the Gaussian filter G, take the second derivative of the Gaussian filter G σ and perform convolution with each pixel point in the region of interest one by one to obtain I XX (σ), I XY (σ), I XZ (σ), I YY (σ), I YZ (σ) and I ZZ (σ);

[0023] Construct Hessian matrices H1 and H2, and the calculation formulas are:

[0024] and

[0025] Calculate the eigenvalues λ1, λ2 of H1 and the eigenvalues λ3, λ4 of H2 and sort them, satisfying |λ1| > |λ2|, |λ3| > |λ4|;

[0026] Calculate the enhancement values E1 and E2 of each pixel point in the region of interest, and the calculation formulas are:

[0027] and

[0028] Calculate the minimum value Emin1, the maximum value Emax of the enhancement value E1 in the region of interest, the minimum value Emin2, and the maximum value Emax2 of the enhancement value E2, and finally calculate the final enhancement value E of each pixel point to obtain the enhanced image. The calculation formula of the final enhancement value E is:

[0029]

[0030] According to the method of the present invention, the first pixel threshold is -1000 to -800, and the second pixel threshold is -500 to -300.

[0031] According to the method of the present invention, extracting preliminary interlobar fissure data from each of the enhanced images includes:

[0032] Perform the following operations on the transverse section slice, sagittal section slice, and coronal section slice of the enhanced image respectively:

[0033] For each pixel point in the region of interest, calculate the sum S of the enhancement values of all points passed by a line segment with a predetermined second length centered on the pixel point E1 , rotate the line segment, and calculate the sum S of the enhancement values in each of the other directions E2 、S E3 、S E4 ……S EN , where N is an integer greater than 1, and count the maximum value S of all the sums of the enhancement values Emax ;

[0034] Take the top predetermined proportion of all the maximum values S in each of the slices as the threshold T, create a binary image, and set the values at the positions where the maximum value S Emax is greater than the threshold T to 1, and the values at the other positions to 0; Emax

[0035] Merge the slices of the binary image into a three-dimensional image;

[0036] Perform an AND operation on the binary image, and extract the largest connected region from the result of the AND operation as the preliminary pulmonary interlobar fissure data.

[0037] According to the method of the present invention, the top predetermined proportion is 2-8%; the second length is 5-15 mm.

[0038] According to the method of the present invention, reconstructing the preliminary pulmonary interlobar fissure data into complete pulmonary interlobar fissure data by using an interpolation algorithm, and then performing pulmonary lobation according to the complete pulmonary interlobar fissure data, includes:

[0039] Traverse each X, Y coordinate in the preliminary pulmonary interlobar fissure data. If the X, Y coordinate corresponds to more than one pulmonary interlobar fissure point: if the distance between the uppermost pulmonary interlobar fissure point and the lowermost pulmonary interlobar fissure point is less than a predetermined first distance, then calculate the average value of the Z coordinates of all the pulmonary interlobar fissure points corresponding to the X, Y coordinate as a new pulmonary interlobar fissure point, otherwise delete all the pulmonary interlobar fissure points corresponding to the X, Y coordinate;

[0040] Convert the adjusted preliminary pulmonary interlobar fissure data into a binary function f(X, Y), and its formula is:

[0041]

[0042] Count all the extreme value points in the binary function f(X, Y) that are neither at the boundary nor at the boundary between the 0-value points and the non-0-value points;

[0043] ​Traverse each point of the adjusted preliminary interlobar fissure data. If the distance between the point and all the interlobar fissure control points and the extreme points is greater than a predetermined second distance, add the point to the interlobar fissure control points.

[0044] Using the biharmonic spline interpolation algorithm, interpolate the extracted interlobar fissure control points to obtain the complete interlobar fissure data.

[0045] Perform lung lobation using the complete interlobar fissure data, and determine the name of each lung lobe according to the positional relationship between the lung lobes to obtain the final lung lobation data.

[0046] According to the method of the present invention, the first distance is 2 - 8 mm, and the second distance is 2 - 8 mm.

[0047] In a second aspect, an embodiment of the present invention provides an automatic lung lobation device based on medical images, which is used to implement the automatic lung lobation method based on medical images as described in any one of the above. The device includes:

[0048] A region acquisition module, which is used to use bronchial data to acquire the regions of interest of each interlobar fissure from the lung medical image to be lobed.

[0049] An image enhancement module, which is used to use the Hessian matrix to enhance each of the regions of interest to obtain a corresponding enhanced image.

[0050] An image extraction module, which is used to extract preliminary interlobar fissure data from each of the enhanced images.

[0051] A lung lobation module, which is used to use the interpolation algorithm to reconstruct and generate complete interlobar fissure data from the preliminary interlobar fissure data, and then perform lung lobation according to the complete interlobar fissure data.

[0052] In a third aspect, an embodiment of the present invention provides an electronic device, which includes a storage medium, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the automatic lung lobation method based on medical images as described in any one of the above.

[0053] The present invention first uses bronchial data to obtain regions of interest (ROIs) of the interlobar fissures of each lung lobe, and uses the Hessian matrix to enhance the interlobar fissures to obtain enhanced images. Then, preliminary interlobar fissure data is extracted from each enhanced image, and the preliminary interlobar fissure data is reconstructed using an interpolation algorithm to generate complete interlobar fissure data. Finally, lung lobation is performed based on the complete interlobar fissure data. The present invention first obtains ROIs according to bronchial data, and then performs segmentation of lung lobes by identifying the interlobar fissures in medical images. It can not only achieve fully automatic lung lobation, but also obtain more accurate lung lobation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 FIG. is a schematic flowchart of an automatic lung lobation method based on medical images provided by an embodiment of the present invention;

[0055] Figure 2 FIG. is a flowchart of a method for obtaining regions of interest using bronchial data in an automatic lung lobation method based on medical images provided by an embodiment of the present invention;

[0056] Figure 3 FIG. is a flowchart of a method for enhancing interlobar fissures in an automatic lung lobation method based on medical images provided by an embodiment of the present invention;

[0057] Figure 4 FIG. is a flowchart of a method for extracting interlobar fissures in an automatic lung lobation method based on medical images provided by an embodiment of the present invention;

[0058] Figure 5 FIG. is a flowchart of a method for reconstructing interlobar fissures in an automatic lung lobation method based on medical images provided by an embodiment of the present invention;

[0059] Figure 6 FIG. is a schematic structural diagram of an automatic lung lobation device based on medical images provided by an embodiment of the present invention;

[0060] Figure 7 FIG. is a schematic structural diagram of an electronic device provided by an embodiment of the present invention;

[0061] Figure 8 FIG. is a schematic hardware structure diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0062] In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0063] It should be noted that the references to "one embodiment", "embodiment", "example embodiment", etc. in this specification mean that the described embodiment may include specific features, structures or characteristics, but not every embodiment must include these specific features, structures or characteristics. In addition, such expressions do not refer to the same embodiment. Further, when combining specific features, structures or characteristics with an embodiment, it has been shown that it is within the knowledge of those skilled in the art to combine such features, structures or characteristics with other embodiments, whether or not explicitly described.

[0064] In addition, in the specification and subsequent claims, certain terms are used to refer to specific components or parts. Those of ordinary skill in the art should understand that manufacturers may use different nouns or terms to refer to the same component or part. The specification and subsequent claims do not use the difference in names as a way to distinguish components or parts, but use the difference in the functions of components or parts as the criterion for distinction. The terms "comprising" and "including" mentioned throughout the specification and subsequent claims are open-ended terms, and should therefore be interpreted as "including but not limited to". In addition, the term "connected" herein includes any direct and indirect electrical connection means. Indirect electrical connection means include connection through other devices.

[0065] Next, with reference to the accompanying drawings, through specific embodiments and their application scenarios, the automatic lung lobation method based on medical images provided by the embodiments of the present invention will be described in detail.

[0066] Figure 1 is a schematic flowchart of the automatic lung lobation method based on medical images provided by the embodiments of the present invention. The method includes the following steps:

[0067] Step S101, using bronchial data, obtain the regions of interest of the interlobar fissures of each lung lobe from the lung medical image to be lobed.

[0068] Optionally, the lung medical image may be a CT image or an MRI image, etc.

[0069] Step S102, using the Hessian matrix, enhance each region of interest to obtain the corresponding enhanced image.

[0070] Step S103, extract the preliminary interlobar fissure data from each enhanced image.

[0071] Optionally, the preliminary interlobar fissure data is recognizable interlobar fissure data.

[0072] Step S104, use the interpolation algorithm to reconstruct the preliminary interlobar fissure data into complete interlobar fissure data, and then perform lung lobation according to the complete interlobar fissure data.

[0073] The main problems to be solved by the present invention are that manual lung lobation is time-consuming and laborious, and the existing automatic and semi-automatic lung lobation methods do not fully utilize CT image information, resulting in inaccurate results. A method for automatic lung lobation based on medical images is proposed. First, the region of interest is obtained according to bronchial data. The interlobar fissures of the lungs are enhanced using the Hessian matrix, and the interlobar fissures are extracted and reconstructed. The lung lobes are segmented by identifying the interlobar fissures in medical images such as CT, which can not only achieve fully automatic lung lobation, but also obtain more accurate lobation results.

[0074] Figure 2 It is a flowchart of the method for obtaining the region of interest using bronchial data in the method for automatic lung lobation based on medical images provided by an embodiment of the present invention. The method includes:

[0075] Step S201, calculate the Euler characteristic values of each discrete point of the bronchus to obtain the bronchial centerline point set.

[0076] Optionally, calculate the 26-neighborhood Euler characteristic values of each discrete point of the bronchus to obtain the bronchial centerline point set.

[0077] Step S202, construct a bronchial tree structure according to the bronchial centerline point set.

[0078] Step S203, automatically mark the bronchial centerlines corresponding to the five lung lobes according to the bronchial tree structure and the physiological characteristics of the bronchi.

[0079] Step S204, calculate the distances between each pixel point in the lung and the five bronchial centerlines, and mark the pixel point with the same mark as the nearest bronchial centerline.

[0080] Step S205, obtain the set of junction points between two different marks, calculate the regression plane using the set of junction points, and expand the regression plane by a predetermined first length as the region of interest for calculating the interlobar fissures.

[0081] Optionally, the range of the first length is 25-35 mm. The first length is preferably 30 mm.

[0082] Step S206, repeat this operation until the regions of interest of each interlobar fissure are calculated, and subsequent operations are performed on the regions of interest of each interlobar fissure respectively.

[0083] Figure 3 It is a flowchart of the method for enhancing the interlobar fissures in the method for automatic lung lobation based on medical images provided by an embodiment of the present invention. The method includes:

[0084] Step S301: Modify the pixel values of the regions in the region of interest where the pixel values are less than the first pixel threshold to the first pixel threshold, and modify the pixel values of the regions where the pixel values are greater than the second pixel threshold to the second pixel threshold.

[0085] Optionally, the range of the first pixel threshold is -1000 to -800, and the range of the second pixel threshold is -500 to -300.

[0086] Preferably, the first pixel threshold is -900 and the second pixel threshold is -400. That is, modify the pixel values of the regions in the CT image in the region of interest where the pixel values are less than -900 to -900; modify the pixel values of the regions where the CT pixel values are greater than -400 to -400.

[0087] Step S302: Construct a Gaussian filter G σ , where σ is the variance of the Gaussian filter G σ , take the second derivative of the Gaussian filter G σ , and perform convolution with each pixel point in the region of interest one by one using the second derivative to obtain I XX (σ), I XY (σ), I XZ (σ), I YY (σ), I YZ (σ) and I ZZ (σ).

[0088] Step S303: Construct Hessian matrices H1 and H2, and the calculation formulas are:

[0089] and

[0090] Calculate the eigenvalues λ1, λ2 of H1 and the eigenvalues λ3, λ4 of H2 and sort them, satisfying |λ1| > |λ2|, |λ3| > |λ4|.

[0091] Step S304: Calculate the enhancement values E1 and E2 of each pixel point in the region of interest, and the calculation formulas are:

[0092] and

[0093] Step S305: Calculate the minimum value Emin1, the maximum value Emax1 of the enhancement value E1 in the region of interest, and the minimum value Emin2, the maximum value Emax2 of the enhancement value E2, and finally calculate the final enhancement value E of each pixel point to obtain the enhanced image. The calculation formula of the final enhancement value E is:

[0094]

[0095] Figure 4It is a flowchart of a method for extracting pulmonary fissures in the automatic lung lobation method based on medical images provided by an embodiment of the present invention, which includes:

[0096] Step S401, perform the following operations on the cross-sectional slices, sagittal slices, and coronal slices of the enhanced image respectively:

[0097] For each pixel point in the region of interest, calculate the sum S of the enhancement values of all points passed by a line segment with a predetermined second length centered on the pixel point E1 , rotate the line segment, and calculate the sum S of the enhancement values in other directions E2 、S E3 、S E4 ……S EN , where N is an integer greater than 1, and count the maximum value S of all the sums of enhancement values Emax . Optionally, the second length is 5-15 mm. The second length is preferably 10 mm.

[0098] For each slice, take the top predetermined proportion of all the maximum values S Emax in this slice as the threshold T, create a binary image, set the values where the maximum value S Emax is greater than the threshold T to 1, and set the values in other places to 0. Optionally, the top predetermined proportion is 2-8%.

[0099] The top predetermined proportion is preferably 5%. That is, for each slice, take the top 5% of all S Emax in this slice as the threshold T, create a binary image, and set the values where S Emax is greater than T to 1, and set the values in other places to 0.

[0100] Step S402, merge the slices of the binary image into a three-dimensional image.

[0101] Step S403, perform an AND operation on the binary images created in the three directions of the cross-section, sagittal plane, and coronal plane in the previous step, and extract the largest connected region from the result of the AND operation as the preliminary pulmonary fissure data.

[0102] Figure 5 It is a flowchart of a method for reconstructing pulmonary fissures in the automatic lung lobation method based on medical images provided by an embodiment of the present invention, which includes:

[0103] Step S501, traverse each X, Y coordinate in the preliminary pulmonary fissure data. If the X, Y coordinate corresponds to more than one pulmonary fissure point: if the distance between the uppermost pulmonary fissure point and the lowermost pulmonary fissure point is less than a predetermined first distance, then calculate the average value of the Z coordinates of all the pulmonary fissure points corresponding to the X, Y coordinate as the new pulmonary fissure point, otherwise delete all the pulmonary fissure points corresponding to the X, Y coordinate.

[0104] Convert the adjusted preliminary interlobar fissure data into a binary function f(X, Y), and its formula is:

[0105]

[0106] Optionally, the first distance is 2-8 mm. The first distance is preferably 5 mm.

[0107] Step S502: Count all the extreme points in the binary function f(X, Y) that are neither at the boundary nor at the boundary between the 0-value points and the non-0-value points.

[0108] Step S503: Traverse each point of the adjusted preliminary interlobar fissure data. If the distance between this point and all the interlobar fissure control points and extreme points is greater than a predetermined second distance, add this point to the interlobar fissure control points.

[0109] Optionally, the second distance is 2-8 mm. The second distance is preferably 5 mm.

[0110] Step S504: Use the biharmonic spline interpolation algorithm to interpolate the extracted interlobar fissure control points into complete interlobar fissure data.

[0111] Step S505: Perform lung lobation using the complete interlobar fissure data, and determine the name of each lung lobe according to the positional relationship between the lung lobes to obtain the final lung lobation data.

[0112] The automatic lung lobation method based on medical images provided by the embodiments of the present invention first uses bronchial data to obtain the regions of interest of each interlobar fissure, and uses the Hessian matrix to enhance the interlobar fissure to obtain an enhanced image. Then, preliminary interlobar fissure data is extracted from each enhanced image, and the preliminary interlobar fissure data is reconstructed into complete interlobar fissure data using the interpolation algorithm, and then lung lobation is performed according to the complete interlobar fissure data. The present invention first obtains the regions of interest according to bronchial data, and then performs the segmentation of lung lobes by identifying the interlobar fissure in the medical image. It can not only achieve fully automatic lung lobation, but also obtain more accurate lung lobation results.

[0113] It should be noted that for the automatic lung lobation method based on medical images provided by the embodiments of the present invention, the execution subject can be an electronic device, an automatic lung lobation device based on medical images, or a control module in the automatic lung lobation device based on medical images for executing the automatic lung lobation method based on medical images. In the embodiments of the present invention, the case where the automatic lung lobation device based on medical images executes the automatic lung lobation method is taken as an example to illustrate the automatic lung lobation device provided by the embodiments of the present invention.

[0114] Figure 6It is a schematic structural diagram of an automatic lung lobation device based on medical images provided by an embodiment of the present invention, which is used to implement the method for segmenting pulmonary blood vessels in medical images as shown in Figures 1 to 5 The automatic lung lobation device 100 based on medical images includes:

[0115] A region acquisition module 101, configured to use bronchial data to acquire regions of interest of the interlobar fissures from the pulmonary medical images to be lobated.

[0116] An image enhancement module 102, configured to use the Hessian matrix to enhance each region of interest to obtain a corresponding enhanced image.

[0117] An image extraction module 103, configured to extract preliminary interlobar fissure data from each enhanced image.

[0118] A lung lobation module 104, configured to use an interpolation algorithm to reconstruct the preliminary interlobar fissure data into complete interlobar fissure data, and then perform lung lobation according to the complete interlobar fissure data.

[0119] Optionally, the region acquisition module 101 includes:

[0120] A bronchial centerline point set calculation sub-module, configured to calculate the Euler characteristic value of each discrete point of the bronchus to obtain a bronchial centerline point set.

[0121] A bronchial tree structure construction sub-module, configured to construct a bronchial tree structure according to the bronchial centerline point set.

[0122] A first marking sub-module, configured to automatically mark the bronchial centerlines corresponding to the five lung lobes according to the bronchial tree structure and the bronchial physiological characteristics.

[0123] A second marking sub-module, configured to calculate the distance between each pixel point in the lung and the five bronchial centerlines, and mark the pixel point with the same mark as the nearest bronchial centerline.

[0124] A region of interest calculation sub-module, configured to obtain a set of joint points between two different marks, calculate a regression plane using the set of joint points, and expand the regression plane by a predetermined first length as the region of interest for calculating the interlobar fissure; preferably, the first length is 25-35 mm.

[0125] Repeat this operation until the regions of interest of each interlobar fissure are calculated, and subsequent operations are performed separately on the regions of interest of each interlobar fissure.

[0126] Optionally, the image enhancement module 102 includes:

[0127] A pixel value modification sub-module, configured to modify the pixel values of the regions in the region of interest where the pixel values are less than the first pixel threshold to the first pixel threshold, and modify the pixel values of the regions where the pixel values are greater than the second pixel threshold to the second pixel threshold. Preferably, the first pixel threshold is -1000 to -800, and the second pixel threshold is -500 to -300.

[0128] A pixel point convolution sub-module, configured to construct a Gaussian filter G σ , where σ is the variance of the Gaussian filter G σ Take the second derivative of the Gaussian filter G σ Perform convolution with each pixel point in the region of interest one by one using the second derivative to obtain I XX (σ), I XY (σ), I XZ (σ), I YY (σ), I YZ (σ) and I ZZ (σ).

[0129] A matrix composition sub-module, configured to compose Hessian matrices H1 and H2, and the calculation formula is:

[0130] and

[0131] Calculate the eigenvalues λ1, λ2 of H1 and the eigenvalues λ3, λ4 of H2 and sort them, satisfying |λ1| > |λ2|, |λ3| > |λ4|.

[0132] A first enhancement value calculation sub-module, configured to calculate the enhancement values E1 and E2 of each pixel point in the region of interest, and the calculation formula is:

[0133] and

[0134] A second enhancement value calculation sub-module, configured to calculate the minimum value Emin1, the maximum value Emax1 of the enhancement value E1 in the region of interest, and the minimum value Emin2, the maximum value Emax2 of the enhancement value E2, and finally calculate the final enhancement value E of each pixel point to obtain an enhanced image. The calculation formula of the final enhancement value E is:

[0135]

[0136] The image extraction module 103 includes:

[0137] A binary image creation sub-module, configured to perform the following operations on the cross-sectional slice, sagittal slice, and coronal slice of the enhanced image respectively:

[0138] For each pixel point in the region of interest, calculate the sum S of the enhancement values of all points passed by a line segment with a predetermined second length centered on the pixel point E1 , rotate the line segment, and calculate the sum S of the enhancement values in each of the remaining directions E2 、S E3 、S E4 ……S EN , where N is an integer greater than 1, and count the maximum value S of all the sums of enhancement values Emax . Preferably, the second length is 5-15 mm

[0139] Take the top predetermined proportion of all the maximum values S in each slice as the threshold T, create a binary image, and set the values where the maximum value S Emax is greater than the threshold T to 1, and the values elsewhere to 0. Preferably, the top predetermined proportion is 2-8%. Emax A three-dimensional image generation sub-module for merging the slices of the binary image into a three-dimensional image

[0140] A preliminary pulmonary fissure data generation sub-module for performing an AND operation on the binary image and extracting the largest connected region from the result of the AND operation as the preliminary pulmonary fissure data

[0141] Optionally, the lung lobation module 104 includes:

[0142] A pulmonary fissure point adjustment sub-module for traversing each X, Y coordinate in the preliminary pulmonary fissure data. If the X, Y coordinate corresponds to more than one pulmonary fissure point: if the distance between the uppermost pulmonary fissure point and the lowermost pulmonary fissure point is less than a predetermined first distance, calculate the average value of the Z coordinates of all the pulmonary fissure points corresponding to the X, Y coordinate as the new pulmonary fissure point; otherwise, delete all the pulmonary fissure points corresponding to the X, Y coordinate. Preferably, the first distance is 2-8 mm

[0143] A binary function conversion sub-module for converting the adjusted preliminary pulmonary fissure data into a binary function f(X, Y), and its formula is:

[0144]

[0145]

[0146] An extreme point sub-module for counting all the extreme points in the binary function f(X, Y) that are neither at the boundary nor at the boundary between the 0-value points and the non-0-value points

[0147] A control point addition sub-module for traversing each point in the adjusted preliminary pulmonary fissure data. If the distance between the point and all the pulmonary fissure control points and extreme points is greater than a predetermined second distance, add the point to the pulmonary fissure control points. Preferably, the second distance is 2-8 mm

[0148] A complete pulmonary fissure data generation sub-module, which is used to interpolate the extracted pulmonary fissure control points into complete pulmonary fissure data by using the biharmonic spline interpolation algorithm.

[0149] A lung lobation sub-module, which is used to perform lung lobation by using the complete pulmonary fissure data, and determine the name of each lung lobe according to the positional relationship between the lung lobes, so as to obtain the final lung lobation data.

[0150] The automatic lung lobation device based on medical images in the embodiments of the present invention can be a device, or a component, an integrated circuit, or a chip in a terminal. The device can be a mobile electronic device or a non-mobile electronic device. Exemplarily, the mobile electronic device can be a mobile phone, a tablet computer, a laptop computer, a palmtop computer, a vehicle-mounted electronic device, a wearable device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA), etc., and the non-mobile electronic device can be a server, a Network Attached Storage (NAS), a personal computer (PC), a television (TV), a teller machine, or a self-service machine, etc. The embodiments of the present invention do not make specific limitations.

[0151] The automatic lung lobation device based on medical images in the embodiments of the present invention can be a device with an operating system. The operating system can be the Android operating system, the iOS operating system, or other possible operating systems. The embodiments of the present invention do not make specific limitations.

[0152] The automatic lung lobation device based on medical images provided in the embodiments of the present invention can implement each process implemented by the method embodiment of automatic lung lobation based on medical images. To avoid repetition, it will not be elaborated here.

[0153] The automatic lung lobation device based on medical images provided in the embodiments of the present invention first uses bronchial data to obtain the regions of interest of each pulmonary fissure, and uses the Hessian matrix to enhance the pulmonary fissure to obtain an enhanced image, then extracts the preliminary pulmonary fissure data from each enhanced image, uses the interpolation algorithm to reconstruct and generate the complete pulmonary fissure data, and then performs lung lobation according to the complete pulmonary fissure data. The present invention first obtains the regions of interest according to the bronchial data, and then segments the lung lobes by identifying the pulmonary fissure in the medical image. It can not only achieve full-automatic lung lobation, but also obtain more accurate lung lobation results.

[0154] Optionally, such asFigure 7 As shown in the figure, an embodiment of the present invention further provides an electronic device 500, including a processor 501, a memory 502, a program or instruction stored on the memory 502 and operable on the processor 501. When the program or instruction is executed by the processor 501, it implements each process of the above-mentioned embodiment of the automatic lung lobation method based on medical images and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0155] Figure 8 It is a schematic diagram of the hardware structure of an electronic device implementing an embodiment of the present invention.

[0156] The electronic device 600 includes, but is not limited to: a radio frequency unit 601, a network module 602, an audio output unit 603, an input unit 604, a sensor 605, a display unit 606, a user input unit 607, an interface unit 608, a memory 609, and a processor 610 and other components.

[0157] It should be understood that in an embodiment of the present invention, the radio frequency unit 601 can be used for receiving and sending information or signals during a call. Specifically, after receiving the downlink data from the base station, it is given to the processor 610 for processing; in addition, the uplink data is sent to the base station. Usually, the radio frequency unit 601 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier, a duplexer, etc. In addition, the radio frequency unit 601 can also communicate with the network and other devices through a wireless communication system.

[0158] The electronic device provides the user with wireless broadband Internet access through the network module 602, such as helping the user to send and receive e-mails, browse web pages, and access streaming media, etc.

[0159] The audio output unit 603 can convert the audio data received by the radio frequency unit 601 or the network module 602 or stored in the memory 609 into an audio signal and output it as sound. Moreover, the audio output unit 603 can also provide an audio output related to a specific function executed by the electronic device 600 (for example, a call signal reception sound, a message reception sound, etc.). The audio output unit 603 includes a speaker, a buzzer, and a receiver, etc.

[0160] The input unit 604 is used for receiving audio or video signals. It should be understood that in an embodiment of the present invention, the input unit 604 may include a Graphics Processing Unit (GPU) 6041 and a microphone 6042. The graphics processor 6041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode.

[0161] The electronic device 600 further includes at least one sensor 605, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor and a proximity sensor. Among them, the ambient light sensor can adjust the brightness of the display panel 6061 according to the brightness of the ambient light, and the proximity sensor can turn off the display panel 6061 and / or the backlight when the electronic device 600 is moved to the ear. As a kind of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity, and can be used to identify the posture of the electronic device (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; the sensor 605 can also include a fingerprint sensor, a pressure sensor, an iris sensor, a molecular sensor, a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, etc., which will not be elaborated here.

[0162] The display unit 606 is used to display information input by the user or information provided to the user. The display unit 606 may include a display panel 6061, and the display panel 6061 can be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc.

[0163] The user input unit 607 can be used to receive input digital or character information, and generate key signal inputs related to the user settings and function control of the electronic device. Specifically, the user input unit 607 includes a touch panel 6071 and other input devices 6072. The touch panel 6071, also known as a touch screen, can collect touch operations of the user on or near it (such as operations of the user using a finger, a stylus, or any suitable object or accessory on or near the touch panel 6071). The touch panel 6071 can include two parts: a touch detection device and a touch controller. The other input devices 6072 can include but are not limited to a physical keyboard, function keys (such as volume control buttons, switch buttons, etc.), a trackball, a mouse, a joystick, which will not be elaborated here.

[0164] The interface unit 608 is an interface for connecting an external device to the electronic device 600. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device with an identification module, an audio input / output (I / O) port, a video I / O port, a headset port, and so on. The interface unit 608 can be used to receive inputs from an external device (such as data information, power, etc.) and transfer the received inputs to one or more components within the electronic device 600 or can be used to transfer data between the electronic device 600 and the external device.

[0165] The memory 609 can be used to store software programs and various data. The memory 609 mainly includes a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function (such as a sound playback function, an image playback function, etc.); the data storage area can store data created according to the use of the mobile phone (such as audio data, phone book, etc.). In addition, the memory 609 can include high-speed random access memory and can also include non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices.

[0166] The processor 610 is the control center of the electronic device, connecting various parts of the entire electronic device through various interfaces and lines. By running or executing software programs and / or modules stored in the memory 609, and by calling data stored in the memory 609, it executes various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole. The processor 610 may include one or more processing units; preferably, the processor 610 may integrate an application processor and a modem processor. Among them, the application processor mainly processes the operating system, user interface, and application programs, etc., and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor may not be integrated into the processor 610.

[0167] Those skilled in the art can understand that the electronic device 600 may further include a power supply (such as a battery) for powering each component. The power supply can be logically connected to the processor 610 through a power management system, so as to realize functions such as management of charging, discharging, and power consumption management through the power management system. Figure 8 The structure of the electronic device shown in the figure does not limit the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements, which will not be elaborated here. In the embodiments of the present invention, the electronic device includes but is not limited to mobile phones, tablet computers, laptop computers, handheld computers, vehicle-mounted terminals, wearable devices (such as bracelets, glasses), and pedometers, etc.

[0168] An embodiment of the present invention further provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, each process of the above-mentioned embodiment of the automatic lung lobation method based on medical images is implemented, and the same technical effects can be achieved. To avoid repetition, it will not be elaborated here.

[0169] Wherein, the processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer-readable storage medium, such as a computer read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disc, etc.

[0170] Another embodiment of the present invention provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor, and the processor is configured to run a program or instruction to implement each process of the above-mentioned embodiment of the automatic lung lobation method based on medical images, and the same technical effects can be achieved. To avoid repetition, it will not be elaborated here.

[0171] It should be understood that the chip mentioned in the embodiment of the present invention may also be referred to as a system-on-chip, a system chip, a chip system, or a system-on-chip.

[0172] It should be noted that in this article, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the phrase "including one..." does not exclude the existence of additional identical elements in the process, method, article or device including that element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present invention is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in a reverse order according to the functions involved. For example, the described methods may be performed in an order different from that described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0173] Through the description of the above embodiments, those skilled in the art can clearly understand that the above method of the embodiment can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions for causing a terminal (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in various embodiments of the present invention.

[0174] Certainly, the present invention may also have other various embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art can make various corresponding changes and deformations according to the present invention, but these corresponding changes and deformations should all fall within the protection scope of the appended claims of the present invention.

Claims

1. An automatic lung lobation method based on medical images, characterized in that The steps include: Using bronchial data, obtaining the regions of interest of each interlobar fissure from the pulmonary medical image to be lobulated; Using the Hessian matrix to enhance each of the regions of interest to obtain corresponding enhanced images; Extracting preliminary interlobar fissure data from each of the enhanced images; Using an interpolation algorithm to reconstruct the preliminary interlobar fissure data into complete interlobar fissure data, and then performing pulmonary lobulation based on the complete interlobar fissure data; The step of using an interpolation algorithm to reconstruct the preliminary interlobar fissure data into complete interlobar fissure data and then performing pulmonary lobulation based on the complete interlobar fissure data includes: Traversing each X, Y coordinate in the preliminary interlobar fissure data. If the X, Y coordinate corresponds to more than one interlobar fissure point: If the distance between the uppermost interlobar fissure point and the lowermost interlobar fissure point is less than a predetermined first distance, then calculate the average value of the Z coordinates of all the interlobar fissure points corresponding to the X, Y coordinate as the new interlobar fissure point; otherwise, delete all the interlobar fissure points corresponding to the X, Y coordinate; Converting the adjusted preliminary interlobar fissure data into a binary function f(X, Y), and its formula is: Counting all the extreme points in the binary function f(X, Y) that are not at the boundary, nor at the boundary between the 0-value points and the non-0-value points; Traversing each point in the adjusted preliminary interlobar fissure data. If the distance between the point and all the interlobar fissure control points and the extreme points is greater than a predetermined second distance; add the point to the interlobar fissure control points; Using the bicubic B-spline interpolation algorithm to interpolate the extracted interlobar fissure control points into the complete interlobar fissure data; Performing pulmonary lobulation using the complete interlobar fissure data, and determining the name of each lobe according to the positional relationship between the lobes to obtain the final pulmonary lobulation data; The step of extracting preliminary interlobar fissure data from each of the enhanced images includes: Performing the following operations respectively on the cross-sectional slice, sagittal slice and coronal slice of the enhanced image: For each pixel point in the region of interest, calculate the sum S of the enhancement values of all points passed by a line segment with a predetermined second length centered on the pixel point E1 , rotate the line segment, and calculate the sum S of the enhancement values in each of the remaining directions E2 、S E3 、S E4 ……S EN , where N is an integer greater than 1, and count the maximum value S of all the sums of the enhancement values Emax ; Take the first predetermined proportion of all the maximum values S in each of the said slices as the threshold T, create a binary image, and set the values where the maximum value S Emax is greater than the threshold T to 1, and the values at other places to 0; Emax ​ Combining the slices of the binary image into a three-dimensional image; Performing an AND operation on the binary image, and extracting the largest connected region from the result of the AND operation as the preliminary interlobar fissure data.

2. The method according to claim 1, wherein The step of using bronchial data to obtain the regions of interest of each interlobar fissure from the pulmonary medical image to be lobulated includes: Calculating the Euler characteristic value of each discrete point of the bronchus to obtain the bronchial centerline point set; Constructing a bronchial tree structure according to the bronchial centerline point set; Automatically marking the bronchial centerlines corresponding to the five lobes according to the bronchial tree structure and bronchial physiological characteristics; Calculating the distance between each pixel point in the lung and the five bronchial centerlines, and making the same mark for the pixel point and the bronchial centerline with the closest distance; Obtain the set of connecting points between two different markers, calculate the regression plane using the set of connecting points, expand the regression plane outward by a predetermined first length as the region of interest for calculating the interlobar fissure of the lung; repeat this operation until the regions of interest for each interlobar fissure of the lung are calculated, and subsequent operations are performed separately on the regions of interest for each interlobar fissure of the lung.

3. The method according to claim 2, wherein The first length is 25 - 35 mm.

4. The method according to any one of claims 1 to 3, characterized in that The use of the Hessian matrix to enhance each of the regions of interest to obtain a corresponding enhanced image includes: Modify the pixel values of the regions in the region of interest where the pixel values are less than the first pixel threshold to the first pixel threshold, and modify the pixel values of the regions where the pixel values are greater than the second pixel threshold to the second pixel threshold; Construct the Gaussian filter G σ , where σ is the variance of the Gaussian filter G σ . Take the second derivative of the Gaussian filter G σ , and perform convolution one by one with each pixel point in the region of interest using the second derivative to obtain I XX (σ), I XY (σ), I XZ (σ), I YY (σ), I YZ (σ) and I ZZ (σ); Construct the Hessian matrices H1 and H2, and the calculation formula is: and Calculate the eigenvalues λ1, λ2 of H1 and the eigenvalues λ 3, , λ4 of H2 and sort them, satisfying |λ1| > |λ2|, |λ 3, | > |λ4|; calculate the enhancement values E1 and E2 of each pixel point in the region of interest, and the calculation formula is: and Calculate the minimum value Emin1 and the maximum value Emax1 of the enhancement value E1 in the region of interest, and the minimum value Emin2 and the maximum value Emax2 of the enhancement value E2, and finally calculate the final enhancement value E of each pixel point to obtain the enhanced image. The calculation formula of the final enhancement value E is:

5. The method according to claim 4, wherein The first pixel threshold is -1000 to -800, and the second pixel threshold is -500 to -300.

6. The method according to claim 1, wherein The pre-specified ratio is 2 - 8%; the second length is 5 - 15 mm.

7. The method according to claim 1, wherein The first distance is 2 - 8 mm, and the second distance is 2 - 8 mm.

8. An automatic lung lobation device based on medical images, used to implement the automatic lung lobation method based on medical images according to any one of claims 1 - 7. The device includes: A region acquisition module, used to obtain the regions of interest of each interlobar fissure of the lung from the medical image of the lung to be lobated using bronchial data; An image enhancement module, used to enhance each of the regions of interest using the Hessian matrix to obtain a corresponding enhanced image; An image extraction module, used to extract preliminary interlobar fissure data from each of the enhanced images; A lung lobation module, used to reconstruct the preliminary interlobar fissure data into complete interlobar fissure data using an interpolation algorithm, and then perform lung lobation according to the complete interlobar fissure data.

9. An electronic device, comprising a storage medium, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the automatic lung lobation method based on medical images according to any one of claims 1 - 7.

Images