Focus positioning method and system, electronic equipment and storage medium

Abstract

The invention provides a focus positioning method and system, electronic equipment and a storage medium. The focus positioning method comprises the following steps: acquiring medical image data of a target tissue containing a focus; performing three-dimensional reconstruction on the target tissue according to the medical image data to obtain a three-dimensional model of the target tissue; performing multi-level segmentation on the target organization according to the three-dimensional model to obtain a multi-level segmentation result of the target organization; and obtaining the accurate position of the focus in the target tissue based on the segmentation result. According to the method, three-dimensional reconstruction is carried out on the basis of the medical image data of the target tissue containing the focus, then the accurate position of the focus in the target tissue is obtained in combination with the multi-level segmentation result of the target tissue, and accurate and rapid multi-dimensional positioning of the focus is achieved.

Description

Technical Field

[0001] The present disclosure relates to the field of medical imaging technology, and in particular to a lesion localization method, system, electronic device and storage medium. Background Art

[0002] In the prior art, doctors usually observe the patient's medical imaging data and, based on their professional knowledge and subjective experience, look for areas that meet the imaging characteristics of the lesion to preliminarily determine the location of the lesion. They then determine the specific range of the lesion's invasion by measuring specified physiological parameters to locate the lesion.

[0003] However, the above method is too dependent on the professional level of doctors and uses manual methods to locate lesions. This is not only inefficient, but also lacks quantitative indicators of lesions to assist in judgment during the lesion positioning process, resulting in limited accuracy in locating the lesions. Summary of the invention

[0004] In order to overcome the above technical problems, the present disclosure provides a lesion localization method, system, electronic device and storage medium.

[0005] The present invention solves the above technical problems through the following technical solutions:

[0006] In a first aspect, the present disclosure provides a method for locating a lesion. The method for locating a lesion comprises:

[0007] Acquiring medical imaging data of a target tissue containing a lesion;

[0008] Performing three-dimensional reconstruction of the target tissue according to the medical image data to obtain a three-dimensional model of the target tissue;

[0009] Performing multi-level segmentation on the target tissue according to the three-dimensional model to obtain multi-level segmentation results of the target tissue;

[0010] The precise position of the lesion in the target tissue is obtained based on the segmentation result.

[0011] Optionally, the step of performing multi-level segmentation on the target tissue according to the three-dimensional model to obtain multi-level segmentation results of the target tissue includes:

[0012] Determine the model contours of the multi-level segmented tissues of the target tissue respectively according to the medical image data;

[0013] The model contour of the segmented tissue is adjusted.

[0014] Optionally, the step of adjusting the model contour of the segmented tissue comprises:

[0015] The model contour of the lower layer segmented tissue is corrected based on the model contour of the upper layer segmented tissue, so that the model contour of the lower layer segmented tissue is consistent with the model contour of the upper layer segmented tissue; wherein the upper layer segmented tissue includes the lower layer segmented tissue.

[0016] Optionally, the step of respectively determining the model contours of the multi-level segmented tissues of the target tissue according to the medical image data comprises:

[0017] Acquire morphological features of the segmented tissues at multiple levels according to the medical imaging data;

[0018] Determine the corresponding two-dimensional boundary of the segmented tissue according to the morphological features;

[0019] The corresponding model contour is determined according to the two-dimensional boundary of the segmented tissue.

[0020] Optionally, the step of acquiring morphological features of the multi-level segmented tissues according to the medical image data includes:

[0021] Obtaining morphological features of the segmented tissue at at least one level according to the medical image data using a deep learning model;

[0022] Among them, the loss value of the deep learning model during the training process is determined according to the number of voxels that differ between the model contour of the upper segmented tissue and the model contour of the lower segmented tissue.

[0023] Optionally, the multi-level segmented tissues include at least two of lung lobes, lung segments and lung subsegments; wherein the lung lobes are the upper-layer segmented tissues of the lung segments and the lung subsegments, and the lung segments are the upper-layer segmented tissues of the lung subsegments.

[0024] Optionally, the step of adjusting the model contour of the segmented tissue further comprises:

[0025] According to the target segmentation result, a three-dimensional mesh surface is established on the three-dimensional model of the target tissue; wherein the target segmentation result includes a model contour of the segmented tissue corresponding to the target level of the target tissue;

[0026] In response to a change in the mesh data of the three-dimensional mesh surface, the target segmentation result is adjusted.

[0027] In a second aspect, the present disclosure provides a lesion localization system. The lesion localization system comprises:

[0028] An acquisition module, used for acquiring medical imaging data of a target tissue containing a lesion;

[0029] A three-dimensional reconstruction module, used to perform three-dimensional reconstruction of the target tissue according to the medical image data to obtain a three-dimensional model of the target tissue;

[0030] A segmentation module, used for performing multi-level segmentation on the target tissue according to the three-dimensional model to obtain multi-level segmentation results of the target tissue;

[0031] A lesion localization module is used to obtain the precise position of the lesion in the target tissue based on the segmentation result.

[0032] Optionally, the segmentation module includes:

[0033] A model segmentation unit, used to determine the model contours of the multi-level segmented tissues of the target tissue respectively according to the medical image data;

[0034] The contour adjustment unit is used to adjust the model contour of the segmented tissue.

[0035] Optionally, the contour adjustment unit is used to correct the model contour of the lower-layer segmented tissue based on the model contour of the upper-layer segmented tissue, so that the model contour of the lower-layer segmented tissue is consistent with the model contour of the upper-layer segmented tissue; wherein the upper-layer segmented tissue includes the lower-layer segmented tissue.

[0036] Optionally, the model segmentation unit is used to obtain morphological features of the segmented tissue at multiple levels based on the medical image data; determine the corresponding two-dimensional boundaries of the segmented tissue based on the morphological features; and determine the corresponding model contour based on the two-dimensional boundaries of the segmented tissue.

[0037] Optionally, the model segmentation unit is further used to obtain morphological features of the segmented tissue at at least one level according to the medical image data using a deep learning model;

[0038] Among them, the loss value of the deep learning model during the training process is determined according to the number of voxels that differ between the model contour of the upper segmented tissue and the model contour of the lower segmented tissue.

[0039] Optionally, the multi-level segmented tissues include at least two of lung lobes, lung segments and lung subsegments; wherein the lung lobes are the upper-layer segmented tissues of the lung segments and the lung subsegments, and the lung segments are the upper-layer segmented tissues of the lung subsegments.

[0040] Optionally, the contour adjustment unit is further used to establish a three-dimensional mesh surface on the three-dimensional model of the target tissue according to the target segmentation result; wherein the target segmentation result includes the model contour of the segmented tissue corresponding to the target level of the target tissue;

[0041] In response to a change in the mesh data of the three-dimensional mesh surface, the target segmentation result is adjusted.

[0042] In a third aspect, the present disclosure provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and for running on the processor, wherein when the processor executes the computer program, the lesion localization method described in the first aspect is implemented.

[0043] In a fourth aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the lesion localization method described in the first aspect is implemented.

[0044] Based on the common sense in the art, the above-mentioned implementation modes can be arbitrarily combined to obtain the preferred embodiments of the present disclosure.

[0045] The positive progressive effect of the present disclosure lies in: the lesion localization method provided by the embodiment of the present disclosure first performs three-dimensional reconstruction based on the medical imaging data of the target tissue containing the lesion, and then combines the multi-level segmentation results of the target tissue to obtain the precise position of the lesion in the target tissue, thereby achieving accurate and rapid multi-dimensional positioning of the lesion. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A schematic diagram of a process of lesion localization method provided by the present disclosure;

[0047] Figure 2 A schematic diagram of a model contour adjustment of a segmented organization provided by the present disclosure;

[0048] Figure 3 A schematic diagram of a process for locating lung lesions provided by the present disclosure;

[0049] Figure 4 A schematic diagram of a multi-level display of a lung lesion provided by the present disclosure;

[0050] Figure 5 A schematic diagram of a module of a lesion localization system provided by the present disclosure;

[0051] Figure 6 A schematic diagram of the structure of an electronic device provided by the present invention. DETAILED DESCRIPTION

[0052] The present disclosure is further described below by way of examples, but the present disclosure is not limited to the scope of the examples.

[0053] It should be noted that if there are descriptions involving "first", "second", etc. in the embodiments of the present disclosure, the descriptions of "first", "second", etc. are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" or "second" may explicitly or implicitly include at least one of the features.

[0054] In addition, the technical solutions between the various implementation methods can be combined with each other, but they must be based on the fact that they can be implemented by ordinary technicians in this field. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this disclosure.

[0055] As a feasible implementation, the lesion localization method provided in the embodiment of the present disclosure is applied to localize lung lesions.

[0056] That is, the target tissue is the lung, and the location of the lesion in the lung is as follows: Figure 1 As shown, it includes steps S1 to S4:

[0057] S1. Acquire medical imaging data of a target tissue containing a lesion.

[0058] In step S1, the medical image data may be a DR (Digital Radiography) plain film of the lungs or a CT (Computed Tomography) image of the lungs. The medical image data includes, on the one hand, the tissue type information collected after tissue contrast imaging, and on the other hand, the imaging features of the lungs.

[0059] S2. Perform three-dimensional reconstruction of the target tissue according to the medical imaging data to obtain a three-dimensional model of the target tissue.

[0060] In step S2, the two-dimensional slice image of the lung obtained from medical imaging data (such as CT, MRI, etc.) is converted into a three-dimensional geometric model or surface model of the lung. Common three-dimensional reconstruction algorithms include voxelization method, surface reconstruction method and body surface hybrid method. According to the specific application requirements and data type, select the appropriate algorithm for reconstruction.

[0061] Exemplarily, a trained deep learning model is used to segment and extract lung tissues and lung lesions such as the left lung, right lung, and blood vessels in the lungs one by one, and then the lungs and lung lesions are three-dimensionally reconstructed to obtain a lung model and a lesion model.

[0062] Compared with manual labeling, the deep learning model uses a powerful convolutional neural network to extract lung tissue features and lesion features from lung image data, which is more accurate and effective. Image segmentation based on the lung tissue features and lesion features extracted from the lung medical image data can restore the three-dimensional features of lung tissue and lung lesions, realize three-dimensional reconstruction of lung tissue and lung lesions, and obtain lung models and lesion models.

[0063] S3. Perform multi-level segmentation on the target tissue according to the three-dimensional model to obtain multi-level segmentation results of the target tissue.

[0064] In step S3, based on the concept of anatomy, the lungs are segmented according to different tissue levels according to the three-dimensional model. The lungs can be divided into segmented tissues such as lobes, segments, and subsegments according to the levels. For the lobe level, the left lung can be divided into 2 lobes, and the right lung can be divided into 3 lobes; for the segment level, each lobe can be divided into multiple segments; for the subsegment level, each segment can also be divided into multiple subsegments.

[0065] The segmentation result of each level is characterized as the model contours of multiple segmented tissues at the level. In order to make the model contours of the segmented tissues more accurate, the model contours of the segmented tissues can be adjusted.

[0066] As a feasible implementation, step S3 includes:

[0067] S31, determining the model contours of the multi-level segmented tissues of the target tissue respectively according to the medical imaging data;

[0068] S32. Adjust the model contour of the segmented tissue.

[0069] As a feasible implementation, step S31 includes:

[0070] S311, acquiring morphological features of multi-level segmented tissues according to medical imaging data;

[0071] S312, determining the two-dimensional boundary of the corresponding segmented tissue according to the morphological features;

[0072] S313, determining the corresponding model contour according to the two-dimensional boundary of the segmented organization.

[0073] Specifically, for lung lobes, interlobar fissure features can be extracted based on medical imaging data of the lungs, and the two-dimensional boundaries between the lung lobes can be determined based on the interlobar fissure features. For lung segments, segmental bronchial morphological features can be extracted based on medical imaging data of the lungs, and the two-dimensional boundaries between lung segments can be determined based on the segmental bronchial morphological features. Similarly, for lung subsegments, subsegmental bronchial morphological features can be extracted based on medical imaging data of the lungs, and the two-dimensional boundaries between lung subsegments can be determined based on the subsegmental bronchial morphological features. Finally, the corresponding model contours are determined based on the two-dimensional boundaries of the lung lobes, lung segments, and lung subsegments.

[0074] The multi-level segmented tissues include at least two of lobes, segments and subsegments. Generally, the morphological features of lobes characterize interlobular fissures, the morphological features of segments characterize the image morphology of segmental bronchi, and the morphological features of subsegments characterize the image morphology of subsegmental bronchi.

[0075] As a feasible implementation, step S311 specifically includes: using a deep learning model to obtain morphological features of at least one level of segmented tissue according to medical imaging data.

[0076] Specifically, the first deep learning model may be used to extract interlobar fissure features based on the medical image data. The second deep learning model may be used to extract segmental bronchial morphological features based on the medical image data. The third deep learning model may be used to extract subsegmental bronchial morphological features based on the medical image data.

[0077] Exemplarily, the left lung is first segmented once based on medical prior knowledge, and the right lung is segmented twice to obtain the two-dimensional boundaries of multiple lobes, and then the interlobar fissure features are extracted based on medical imaging data to assist in adjusting the two-dimensional boundaries of the lobes, so that the two-dimensional boundaries of the lobes are more accurate. The LDA (Latent Dirichlet Allocation) model is combined with the PCA (Principal Component Analysis) model to form a deep learning model, and the deep learning model of LDA combined with PCA is used to extract the morphological characteristics of the bronchial segments and the morphological characteristics of the subordinate branches of the pulmonary segments, and the pulmonary segments and the subordinate branches of the pulmonary segments are accurately distinguished and identified based on the morphological characteristics of the bronchial segments and the morphological characteristics of the subordinate branches of the pulmonary segments, and then the two-dimensional boundaries of the pulmonary segments can be determined based on the morphological characteristics of the bronchial segments, and the two-dimensional boundaries of the subordinate branches of the pulmonary segments can be determined based on the morphological characteristics of the bronchial segments, and the two-dimensional boundaries of the subordinate branches of the pulmonary segments can be determined based on the morphological characteristics of the subordinate branches of the pulmonary segments. The corresponding deep learning model can also be used to propose the morphological characteristics of the lung subsegmental bronchi, and the two-dimensional boundaries of the lung subsegmental bronchi can be finally determined based on the morphological characteristics of the lung subsegmental bronchi and the two-dimensional boundaries of the subordinate branches.

[0078] In addition, in order to optimize the accuracy of the above-mentioned deep learning model in extracting morphological features, a multidimensional loss function can also be constructed for regression training. The loss value is calculated according to the difference between the model contour of the upper segmented tissue and the model contour of the lower segmented tissue to improve the deep learning model's ability to extract the morphological features of the segmented tissue.

[0079] As a feasible implementation method, the loss value of the deep learning model during the training process can be determined according to the number of voxels that differ between the model contour of the upper-layer segmented tissue and the model contour of the lower-layer segmented tissue.

[0080] Among them, the lung lobe is the upper segmented tissue of the lung segment and the lung subsegment, and the lung segment is the upper segmented tissue of the lung subsegment. It should be understood that the lung can be divided into the left lung and the right lung, and the left lung and the right lung can also be understood as the upper segmented tissue of the lung lobe.

[0081] Specifically, the first loss value of the first deep learning model during the training process is determined according to the first voxel number of the difference between the model contour of the lung lobe and the model contour of the corresponding lung. The second loss value of the second deep learning model during the training process is determined according to the first voxel number of the difference between the model contour of the lung lobe and the model contour of the lung segment. The third loss value of the third deep learning model during the training process is determined according to the third voxel number of the difference between the model contour of the lung segment and the model contour of the lung subsegment.

[0082] Exemplarily, the total number of pixels N of a certain lobe is calculated, and the total number of pixels n1 that the multiple lung segments corresponding to the lobe exceed the total number of pixels n1 of the lobe is calculated, and the total number of pixels n2 that the multiple lung segments corresponding to the lobe lack compared to the lobe is calculated. Construct a multidimensional loss function S = (n1 + n2) / N. Iteratively train the above deep learning model and calculate the loss value according to the multidimensional loss function until the loss value is less than a preset threshold.

[0083] As a feasible implementation, step S32 includes:

[0084] The model contour of the lower layer segmented tissue is corrected based on the model contour of the upper layer segmented tissue, so that the model contour of the lower layer segmented tissue is consistent with the model contour of the upper layer segmented tissue; wherein the upper layer segmented tissue includes the lower layer segmented tissue.

[0085] Among them, the lung lobe is the upper segmented tissue of the lung segment and the lung subsegment, so the model contour of the lung segment or lung subsegment corresponding to the lung lobe can be corrected based on the model contour of the lung lobe. Similarly, the lung segment is the model contour of the lung subsegment, so the model contour of the lung subsegment corresponding to the lung segment can also be corrected according to the model contour of the lung segment. It should be understood that the lung can also be understood as the upper segmented tissue of the lung lobe, so the model contour of the lung lobe corresponding to the lung can also be corrected based on the model contour of the lung.

[0086] Specifically, for the excess area of ​​the model contour of the lower segmented tissue compared to the model contour of the upper segmented tissue, the excess area is cut to correct the model contour of the lower segmented tissue. For the missing area of ​​the model contour of the lower segmented tissue compared to the model contour of the upper segmented tissue, the missing area is filled into the lower segmented tissue to expand the model contour of the lower segmented tissue.

[0087] For example, see Figure 2 , lobe 1 is used as the upper segmented tissue of segment 1, segment 2 and segment 3 respectively, and the excess areas of the model contours of segment 1, lobe 2 and lobe 3 compared with the model contour of lobe 1 are obtained respectively, and cut, and the missing areas of the model contours of segment 1, lobe 2 and lobe 3 compared with the model contour of lobe 1 are obtained respectively, and the missing areas are filled into segment 1, lobe 2 and lobe 3 respectively to expand the model contours of segment 1, lobe 2 and lobe 3.

[0088] Figure 2 The dotted lines in the figure are used to represent an example of filling the missing regions into segment 1, lobe 2, and lobe 3. That is, when filling the missing regions into segment 1, lobe 2, and lobe 3, respectively, the filling can be performed according to the direction of the original model contours of segment 1, lobe 2, and lobe 3.

[0089] As another feasible implementation, step S32 further includes:

[0090] According to the target segmentation result, a three-dimensional mesh surface is established on the three-dimensional model of the target tissue; wherein the target segmentation result includes a model contour of the segmented tissue of the target tissue corresponding to the target level;

[0091] In response to changes in mesh data of the three-dimensional mesh surface, a target segmentation result is adjusted.

[0092] Specifically, by performing subtle translation, rotation, dragging deformation or smoothing operations on the 3D mesh surface, the surface of the 3D model can be made smoother and more refined, and better conform to the real anatomical structure, and the segmentation results at different levels can be fine-tuned.

[0093] Translation and rotation can be achieved by performing spatial matrix transformation on the 3D points and normal vectors of the constructed 3D mesh surface. Smoothing will refit the points on the 3D mesh surface. Dragging generates a new 3D mesh surface based on the direction vector of the push and drag and the set of original points, achieving fine-tuning of the target segmentation results.

[0094] Exemplarily, according to the target segmentation result, a three-dimensional mesh surface is constructed on the three-dimensional model of the target tissue. The three-dimensional mesh surface is a three-dimensional structure composed of a series of vertices, edges, and faces. Each vertex represents a spatial point, and each face is formed by a set of connected vertices, which define the segmentation result of the target level. The deformation of the target segmentation result is achieved by adjusting the position of the vertex. This can change the mesh data of the three-dimensional mesh surface through operations such as translation, rotation, or scaling according to user input or previously calculated information.

[0095] Take building a 3D mesh surface on a 3D model of the lungs as an example:

[0096] According to the segmentation results at the lung lobe level, a three-dimensional mesh surface can be applied between different lung lobe tissues to adjust the model contour of the lung lobe tissue.

[0097] According to the segmentation results at the lung segment level, a three-dimensional mesh surface can be applied between different lung segment tissues to adjust the model contour of the lung segment tissue.

[0098] Similarly, according to the segmentation results at the lung subsegment level, a three-dimensional mesh surface can be applied between different lung subsegment tissues to adjust the model contour of the lung subsegment.

[0099] Based on this implementation, the target segmentation result can be adjusted by operating the three-dimensional mesh surface to make it more consistent with the actual anatomical structure.

[0100] S4. Obtain the precise location of the lesion in the target tissue based on the segmentation results.

[0101] In step S4, quantitative indicators of the lesion can be obtained according to the segmentation results, and the precise position of the lesion in the target tissue can be expressed through multi-level display and quantitative indicators.

[0102] Specifically, the quantitative index includes at least one of the following indicators: basic information of the lesion, size data of the lesion, image parameters of the lesion, and the invasion proportion of the lesion in the invaded segmented tissue.

[0103] Exemplarily, the basic information of the lesion may include the name or number of the lesion. The size data of the lesion may include volume parameters, three-dimensional parameters (width, length and depth), major diameter and vertical major diameter. The imaging parameters of the lesion may include the minimum CT value, maximum CT value and average CT value, maximum cross-sectional area and CTR ratio (Consolidation Tumor Ratio, solid component ratio) of the lesion, etc. The invasion ratio of the lesion in the invaded segmented tissue may include the ratio of the segmented tissues of each different level of the lesion.

[0104] The above-mentioned lesion localization method first performs three-dimensional reconstruction based on the medical imaging data of the target tissue containing the lesion to obtain a three-dimensional model of the target tissue containing the lesion, and then performs multi-level segmentation of the target tissue in combination with the three-dimensional model to obtain multi-level segmentation results of the target tissue, and finally obtains the precise position of the lesion in the target tissue based on the multi-level segmentation results of the target tissue, thereby achieving accurate and rapid multi-dimensional positioning of the lesion.

[0105] In a specific application scenario, the process of locating lung lesions can be seen in Figure 3 ,include:

[0106] First, lung tissue and lung lesions are segmented from the medical imaging data of the lungs, and three-dimensional reconstruction is performed to obtain the lung model and lesion model respectively.

[0107] Then, the lung tissue is multi-level segmented according to the lung three-dimensional model to obtain segmentation results at the lung lobe level, the lung segment level and the lung sub-segment level. The model contour of the lung segment tissue is adjusted based on the model contour of the lung lobe tissue, and the model contour of the lung sub-segment tissue is adjusted based on the adjusted model contour of the lung segment tissue.

[0108] In addition, a three-dimensional mesh surface is established on the three-dimensional model of the lung tissue according to the target segmentation result, and the target segmentation result is finely adjusted by dragging, translating, rotating, smoothing and other operations on the three-dimensional mesh surface.

[0109] Finally, based on the multi-level segmentation results of the lung tissue, the precise location of the lung lesions in the lung tissue is obtained.

[0110] In addition, the specific location of lung lesions in different levels of lung tissue and the quantitative indicators of lung lesions can be displayed in a graded manner.

[0111] Each lung lobe can be displayed separately, or its subordinate lung segments can be displayed. Similarly, its subordinate lung subsegments can be displayed. Due to the subordinate relationship among lung lobes, lung segments, and lung subsegments, segmentation results at different levels can be displayed at the same time, but segmented organizations with subordinate relationships cannot be displayed at the same time.

[0112] For example, the user can choose to expand a certain segmented tissue according to actual needs, so as to obtain more detailed positioning data of the lung lesions and realize multi-dimensional hybrid positioning of the lung lesions.

[0113] For example, see Figure 4 Based on the lung image data, it is determined that there are 3 lung lesions, namely lung lesion 1, lung lesion 2 and lung lesion 3. Lung lesion 1 is located on lobe 1, lung lesion 2 is located on lobe 2, and lung lesion 3 is located on lobe 3.

[0114] Based on the 3D model of the lung tissue and the 3D model of the lung lesion that have been displayed, the user can expand the lung lobe 2 according to actual needs to display the multiple lung segments contained in the lung lobe 2, and can directly check that the lung lesion 2 is located on the lung segment 2. Similarly, the user can expand the lung lobe 3 and then expand the lung segment 3 to display the multiple lung subsegments contained in the lung segment 3, and can directly check that the lung lesion 3 is located on the lung subsegment 3, so as to determine the precise location of the lung lesion and realize the multi-dimensional mixed positioning of the lung lesion.

[0115] Based on the same concept, the present disclosure also provides the following Figure 5 The lesion localization system shown is used to locate lung lesions.

[0116] That is, the target tissue is the lung, and locating the lesion in the lung specifically includes:

[0117] The acquisition module 501 is used to acquire medical image data of a target tissue including a lesion.

[0118] For the acquisition module 501, the medical image data may be a DR (Digital Radiography) plain film of the lungs or a CT (Computed Tomography) image of the lungs. The medical image data includes the tissue type information collected after tissue contrast imaging and the imaging features of the lungs.

[0119] The three-dimensional reconstruction module 502 is used to perform three-dimensional reconstruction of the target tissue according to the medical image data to obtain a three-dimensional model of the target tissue.

[0120] The 3D reconstruction module 502 can convert the 2D slice images of the lungs obtained from medical imaging data (such as CT, MRI, etc.) into a 3D geometric model or surface model of the lungs. Common 3D reconstruction algorithms include voxelization method, surface reconstruction method and body surface hybrid method, etc. According to the specific application requirements and data types, a suitable algorithm is selected for reconstruction.

[0121] Exemplarily, a trained deep learning model is used to segment and extract lung tissues and lung lesions such as the left lung, right lung, and blood vessels in the lungs one by one, and then the lung tissues and lung lesions are three-dimensionally reconstructed to obtain a lung model and a lesion model.

[0122] Compared with manual labeling, the deep learning model uses a powerful convolutional neural network to extract lung tissue features and lesion features from lung image data, which is more accurate and effective. Image segmentation based on the lung tissue features and lesion features extracted from the lung medical image data can restore the three-dimensional features of lung tissue and lung lesions, realize three-dimensional reconstruction of lung tissue and lung lesions, and obtain lung models and lesion models.

[0123] The segmentation module 503 is used to perform multi-level segmentation on the target tissue according to the three-dimensional model to obtain multi-level segmentation results of the target tissue.

[0124] The segmentation module 503 segments the lungs according to different tissue levels based on the concept of anatomy and the three-dimensional model. The lungs can be divided into segmented tissues such as lobes, segments, and subsegments according to the levels. For the lobe level, the left lung can be divided into 2 lobes, and the right lung can be divided into 3 lobes; for the segment level, each lobe can be divided into multiple segments; for the subsegment level, each segment can also be divided into multiple subsegments.

[0125] The segmentation result of each level is characterized as the model contours of multiple segmented tissues at the level. In order to make the model contours of the segmented tissues more accurate, the model contours of the segmented tissues can be adjusted.

[0126] As a feasible implementation, the segmentation module 503 includes:

[0127] A model segmentation unit, used to determine the model contours of the multi-level segmented tissues of the target tissue according to the medical imaging data;

[0128] The contour adjustment unit is used to adjust the model contour of the segmented tissue.

[0129] As a feasible implementation, the model segmentation unit is used to obtain the morphological features of multi-level segmented tissues based on medical imaging data, determine the two-dimensional boundaries of the corresponding segmented tissues based on the morphological features, and determine the corresponding model contour based on the two-dimensional boundaries of the segmented tissues.

[0130] Specifically, for lung lobes, interlobar fissure features can be extracted based on medical imaging data of the lungs, and the two-dimensional boundaries between the lung lobes can be determined based on the interlobar fissure features. For lung segments, segmental bronchial morphological features can be extracted based on medical imaging data of the lungs, and the two-dimensional boundaries between lung segments can be determined based on the segmental bronchial morphological features. Similarly, for lung subsegments, subsegmental bronchial morphological features can be extracted based on medical imaging data of the lungs, and the two-dimensional boundaries between lung subsegments can be determined based on the subsegmental bronchial morphological features. Finally, the corresponding model contours are determined based on the two-dimensional boundaries of the lung lobes, lung segments, and lung subsegments.

[0131] The multi-level segmented tissues include at least two of lobes, segments and subsegments. Generally, the morphological features of lobes characterize interlobular fissures, the morphological features of segments characterize the image morphology of segmental bronchi, and the morphological features of subsegments characterize the image morphology of subsegmental bronchi.

[0132] As a feasible implementation method, the model segmentation unit is specifically used to obtain morphological features of at least one level of segmented tissue according to medical imaging data using a deep learning model.

[0133] Specifically, the first deep learning model may be used to extract interlobar fissure features based on the medical image data. The second deep learning model may be used to extract segmental bronchial morphological features based on the medical image data. The third deep learning model may be used to extract subsegmental bronchial morphological features based on the medical image data.

[0134] Exemplarily, the left lung is first segmented once based on medical prior knowledge, and the right lung is segmented twice to obtain the two-dimensional boundaries of multiple lobes, and then the interlobar fissure features are extracted based on medical imaging data to assist in adjusting the two-dimensional boundaries of the lobes, so that the two-dimensional boundaries of the lobes are more accurate. The LDA (Latent Dirichlet Allocation) model is combined with the PCA (Principal Component Analysis) model to form a deep learning model, and the deep learning model of LDA combined with PCA is used to extract the morphological characteristics of the bronchial segments and the morphological characteristics of the subordinate branches of the pulmonary segments, and the pulmonary segments and the subordinate branches of the pulmonary segments are accurately distinguished and identified based on the morphological characteristics of the bronchial segments and the morphological characteristics of the subordinate branches of the pulmonary segments, and then the two-dimensional boundaries of the pulmonary segments can be determined based on the morphological characteristics of the bronchial segments, and the two-dimensional boundaries of the subordinate branches of the pulmonary segments can be determined based on the morphological characteristics of the bronchial segments, and the two-dimensional boundaries of the subordinate branches of the pulmonary segments can be determined based on the morphological characteristics of the subordinate branches of the pulmonary segments. The corresponding deep learning model can also be used to propose the morphological characteristics of the lung subsegmental bronchi, and the two-dimensional boundaries of the lung subsegmental bronchi can be finally determined based on the morphological characteristics of the lung subsegmental bronchi and the two-dimensional boundaries of the subordinate branches.

[0135] In addition, in order to optimize the accuracy of the above-mentioned deep learning model in extracting morphological features, a multidimensional loss function can also be constructed for regression training. The loss value is calculated according to the difference between the model contour of the upper segmented tissue and the model contour of the lower segmented tissue to improve the deep learning model's ability to extract the morphological features of the segmented tissue.

[0136] As a feasible implementation method, the loss value of the deep learning model during the training process can also be determined according to the number of voxels that differ between the model contour of the upper-layer segmented tissue and the model contour of the lower-layer segmented tissue.

[0137] Among them, the lung lobe is the upper segmented tissue of the lung segment and the lung subsegment, and the lung segment is the upper segmented tissue of the lung subsegment. It should be understood that the lung can be divided into the left lung and the right lung, and the left lung and the right lung can also be understood as the upper segmented tissue of the lung lobe.

[0138] Specifically, the first loss value of the first deep learning model during the training process is determined according to the first voxel number of the difference between the model contour of the lung lobe and the model contour of the corresponding lung. The second loss value of the second deep learning model during the training process is determined according to the first voxel number of the difference between the model contour of the lung lobe and the model contour of the lung segment. The third loss value of the third deep learning model during the training process is determined according to the third voxel number of the difference between the model contour of the lung segment and the model contour of the lung subsegment.

[0139] Exemplarily, the total number of pixels N of a certain lobe is calculated, and the total number of pixels n1 that the multiple lung segments corresponding to the lobe exceed the total number of pixels n1 of the lobe is calculated, and the total number of pixels n2 that the multiple lung segments corresponding to the lobe lack compared to the lobe is calculated. Construct a multidimensional loss function S = (n1 + n2) / N. Iteratively train the above deep learning model and calculate the loss value according to the multidimensional loss function until the loss value is less than a preset threshold.

[0140] As a feasible implementation, the contour adjustment unit is used to correct the model contour of the lower segmented tissue based on the model contour of the upper segmented tissue, so that the model contour of the lower segmented tissue is consistent with the model contour of the upper segmented tissue; wherein the upper segmented tissue includes the lower segmented tissue.

[0141] Among them, the lung lobe is the upper segmented tissue of the lung segment and the lung subsegment, so the model contour of the lung segment or lung subsegment corresponding to the lung lobe can be corrected based on the model contour of the lung lobe. Similarly, the lung segment is the model contour of the lung subsegment, so the model contour of the lung subsegment corresponding to the lung segment can also be corrected according to the model contour of the lung segment. It should be understood that the lung can also be understood as the upper segmented tissue of the lung lobe, so the model contour of the lung lobe corresponding to the lung can also be corrected based on the model contour of the lung.

[0142] Specifically, for the excess area of ​​the model contour of the lower segmented tissue compared to the model contour of the upper segmented tissue, the excess area is cut to correct the model contour of the lower segmented tissue. For the missing area of ​​the model contour of the lower segmented tissue compared to the model contour of the upper segmented tissue, the missing area is filled into the lower segmented tissue to expand the model contour of the lower segmented tissue.

[0143] For example, see Figure 2, lobe 1 is used as the upper segmented tissue of segment 1, segment 2 and segment 3 respectively, and the excess areas of the model contours of segment 1, lobe 2 and lobe 3 compared with the model contour of lobe 1 are obtained respectively, and cut, and the missing areas of the model contours of segment 1, lobe 2 and lobe 3 compared with the model contour of lobe 1 are obtained respectively, and the missing areas are filled into segment 1, lobe 2 and lobe 3 respectively to expand the model contours of segment 1, lobe 2 and lobe 3.

[0144] Figure 2 The dotted lines in the figure are used to represent an example of filling the missing regions into segment 1, lobe 2, and lobe 3. That is, when filling the missing regions into segment 1, lobe 2, and lobe 3, respectively, the filling can be performed according to the direction of the original model contours of segment 1, lobe 2, and lobe 3.

[0145] As another feasible implementation, the contour adjustment unit is also used to establish a three-dimensional mesh surface on the three-dimensional model of the target tissue based on the target segmentation result; wherein the target segmentation result includes the model contour of the segmented tissue of the target tissue corresponding to the target level; and the target segmentation result is adjusted in response to the change of the mesh data of the three-dimensional mesh surface.

[0146] Specifically, by performing subtle translation, rotation, dragging deformation or smoothing operations on the 3D mesh surface, the surface of the 3D model can be made smoother and more refined, and better conform to the real anatomical structure, and the segmentation results at different levels can be fine-tuned.

[0147] Translation and rotation can be achieved by performing spatial matrix transformation on the 3D points and normal vectors of the constructed 3D mesh surface. Smoothing will refit the points on the 3D mesh surface. Dragging generates a new 3D mesh surface based on the direction vector of the push and drag and the set of original points, achieving fine-tuning of the target segmentation results.

[0148] Exemplarily, according to the target segmentation result, a three-dimensional mesh surface is constructed on the three-dimensional model of the target tissue. The three-dimensional mesh surface is a three-dimensional structure composed of a series of vertices, edges, and faces. Each vertex represents a spatial point, and each face is formed by a set of connected vertices, which define the segmentation result of the target level. The deformation of the target segmentation result is achieved by adjusting the position of the vertex. This can change the mesh data of the three-dimensional mesh surface through operations such as translation, rotation, or scaling according to user input or previously calculated information.

[0149] Take building a 3D mesh surface on a 3D model of the lungs as an example:

[0150] According to the segmentation results at the lung lobe level, a three-dimensional mesh surface can be applied between different lung lobe tissues to adjust the model contour of the lung lobe tissue.

[0151] According to the segmentation results at the lung segment level, a three-dimensional mesh surface can be applied between different lung segment tissues to adjust the model contour of the lung segment tissue.

[0152] Similarly, according to the segmentation results at the lung subsegment level, a three-dimensional mesh surface can be applied between different lung subsegment tissues to adjust the model contour of the lung subsegment.

[0153] Based on this implementation, the target segmentation result can be adjusted by operating the three-dimensional mesh surface to make it more consistent with the actual anatomical structure.

[0154] The lesion localization module 504 is used to obtain the precise location of the lesion in the target tissue based on the segmentation result.

[0155] The lesion localization module 504 can obtain quantitative indicators of the lesion according to the segmentation results, and express the precise position of the lesion in the target tissue through multi-level display and quantitative indicators.

[0156] Specifically, the quantitative index includes at least one of the following indicators: basic information of the lesion, size data of the lesion, image parameters of the lesion, and the invasion proportion of the lesion in the invaded segmented tissue.

[0157] Exemplarily, the basic information of the lesion may include the name or number of the lesion. The size data of the lesion may include volume parameters, three-dimensional parameters (width, length and depth), major diameter and vertical major diameter. The imaging parameters of the lesion may include the minimum CT value, maximum CT value and average CT value, maximum cross-sectional area and CTR ratio (Consolidation Tumor Ratio, solid component ratio) of the lesion, etc. The invasion ratio of the lesion in the invaded segmented tissue may include the ratio of the segmented tissues of each different level of the lesion.

[0158] The above-mentioned lesion localization system first performs three-dimensional reconstruction based on the medical imaging data of the target tissue containing the lesion to obtain a three-dimensional model of the target tissue containing the lesion, and then performs multi-level segmentation of the target tissue in combination with the three-dimensional model to obtain multi-level segmentation results of the target tissue, and finally obtains the precise position of the lesion in the target tissue based on the multi-level segmentation results of the target tissue, thereby achieving accurate and rapid multi-dimensional positioning of the lesion.

[0159] Figure 6 The structure of one electronic device disclosed in the present invention is shown. The electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the above-mentioned lesion localization method is implemented. Figure 6 The electronic device 60 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0160] like Figure 6 As shown, the electronic device 60 may also be in the form of a general-purpose computing device, for example, it may be a server device. The components of the electronic device 60 may include, but are not limited to: at least one processor 61, at least one memory 62, and a bus 63 connecting different system components (including the memory 62 and the processor 61).

[0161] The bus 63 includes a data bus, an address bus, and a control bus.

[0162] The memory 62 may include a volatile memory, such as a random access memory (RAM) 621 and / or a cache memory 622 , and may further include a read-only memory (ROM) 623 .

[0163] The memory 62 may also include a program / utility 625 having a set (at least one) of program modules 624, such program modules 624 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0164] The processor 61 executes various functional applications and data processing by running the computer programs stored in the memory 62, such as the lesion localization method disclosed above.

[0165] The electronic device 60 may also communicate with one or more external devices 64 (e.g., keyboards, pointing devices, etc.). Such communication may be performed via an input / output (I / O) interface 65. Furthermore, the model generating device 60 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 66. Figure 6 As shown, the network adapter 66 communicates with other modules of the model-generated device 60 via the bus 63. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with the model-generated device 60, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, RAID (disk array) systems, tape drives, and data backup storage systems, etc.

[0166] It should be noted that although several units / modules or sub-units / modules of the electronic device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided into multiple units / modules to be embodied.

[0167] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon, and when the program is executed by a processor, the above-mentioned lesion localization method is implemented.

[0168] The readable storage medium may include but is not limited to: a portable disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical storage device, a magnetic storage device or any suitable combination of the above.

[0169] In a possible implementation manner, the present disclosure may also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute and implement the above-mentioned lesion localization method.

[0170] Among them, the program code for executing the present disclosure can be written in any combination of one or more programming languages, and the program code can be executed completely on the user device, partially on the user device, as an independent software package, partially on the user device and partially on a remote device, or completely on the remote device.

[0171] Although the specific embodiments of the present disclosure are described above, those skilled in the art should understand that this is only an example, and the protection scope of the present disclosure is defined by the appended claims. Those skilled in the art may make various changes or modifications to these embodiments without departing from the principles and essence of the present disclosure, but these changes and modifications all fall within the protection scope of the present disclosure.

Claims

1. A method for locating a lesion, characterized in that: The lesion localization method comprises: Acquiring medical imaging data of a target tissue containing a lesion; Performing three-dimensional reconstruction of the target tissue according to the medical image data to obtain a three-dimensional model of the target tissue; Performing multi-level segmentation on the target tissue according to the three-dimensional model to obtain multi-level segmentation results of the target tissue; The precise position of the lesion in the target tissue is obtained based on the segmentation result.

2. The method for locating a lesion according to claim 1, characterized in that: The step of performing multi-level segmentation on the target tissue according to the three-dimensional model to obtain multi-level segmentation results of the target tissue includes: Determine the model contours of the multi-level segmented tissues of the target tissue respectively according to the medical image data; The model contour of the segmented tissue is adjusted.

3. The method for locating a lesion according to claim 2, characterized in that: The step of adjusting the model contour of the segmented tissue comprises: The model contour of the lower layer segmented tissue is corrected based on the model contour of the upper layer segmented tissue, so that the model contour of the lower layer segmented tissue is consistent with the model contour of the upper layer segmented tissue; wherein the upper layer segmented tissue includes the lower layer segmented tissue.

4. The method for locating a lesion according to claim 2, characterized in that: The step of respectively determining the model contours of the multi-level segmented tissues of the target tissue according to the medical image data comprises: Acquire morphological features of the segmented tissues at multiple levels according to the medical imaging data; Determine the corresponding two-dimensional boundary of the segmented tissue according to the morphological features; The corresponding model contour is determined according to the two-dimensional boundary of the segmented tissue.

5. The method for locating a lesion according to claim 4, characterized in that: The step of acquiring the morphological features of the multi-level segmented tissues according to the medical image data comprises: Obtaining morphological features of the segmented tissue at at least one level according to the medical image data using a deep learning model; Among them, the loss value of the deep learning model during the training process is determined according to the number of voxels that differ between the model contour of the upper segmented tissue and the model contour of the lower segmented tissue.

6. The method for locating a lesion according to claim 5, characterized in that: The multi-level segmented tissues include at least two of lung lobes, lung segments and lung subsegments; wherein the lung lobes are the upper-layer segmented tissues of the lung segments and the lung subsegments, and the lung segments are the upper-layer segmented tissues of the lung subsegments.

7. The method for locating a lesion according to any one of claims 2 to 6, characterized in that: The step of adjusting the model contour of the segmented tissue also includes: According to the target segmentation result, a three-dimensional mesh surface is established on the three-dimensional model of the target tissue; wherein the target segmentation result includes a model contour of the segmented tissue corresponding to the target level of the target tissue; In response to a change in the mesh data of the three-dimensional mesh surface, the target segmentation result is adjusted.

8. A lesion localization system, characterized in that: The lesion localization system comprises: An acquisition module, used for acquiring medical imaging data of a target tissue containing a lesion; A three-dimensional reconstruction module, used to perform three-dimensional reconstruction of the target tissue according to the medical image data to obtain a three-dimensional model of the target tissue; A segmentation module, used for performing multi-level segmentation on the target tissue according to the three-dimensional model to obtain multi-level segmentation results of the target tissue; A lesion localization module is used to obtain the precise position of the lesion in the target tissue based on the segmentation result.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and used to run on the processor, characterized in that: When the processor executes the computer program, it implements the lesion localization method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the lesion localization method as described in any one of claims 1-7 is implemented.

Images