Lung image three-dimensional reconstruction optimization method and device, electronic device and storage medium

Abstract

The present invention discloses a lung image three-dimensional reconstruction optimization method and device, electronic device and storage medium, which uses a trained fracture repair model to perform vascular repair processing on a target area image in a lung image to obtain a repaired target area image; determines the number of adjacent nodes of each node in a first vascular skeleton line in the repaired target area image; determines the first type of nodes and the second type of nodes in the first vascular skeleton line according to the number of adjacent nodes of each node; determines the vascular skeleton line segments between adjacent first type nodes and second type nodes as vascular branches; calculates the branch length corresponding to the vascular branch; determines the vascular branch with a branch length less than a preset length threshold as a target branch; and performs masking processing on the target branch to obtain a target area image. By masking the vascular branches with branch length less than a preset length threshold in the repaired lung image, the possibility of false positives is reduced, and the optimization of the lung image is achieved.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to a method and device for optimizing three-dimensional reconstruction of lung images, an electronic device, and a storage medium. Background Art

[0002] In the field of modern medical imaging, accurate three-dimensional reconstruction of the internal organ structure of the human body plays an increasingly important role in clinical diagnosis, surgical planning and medical research. Among them, the three-dimensional reconstruction technology of the pulmonary bronchi or pulmonary arteries and veins plays a key role in the early diagnosis of lung diseases and the formulation of personalized treatment plans. The three-dimensional reconstruction of the pulmonary bronchi or pulmonary arteries and veins not only plays an important guiding role for clinicians in diagnosis and treatment decisions, but also provides researchers with an opportunity to deeply explore the relationship between the structure and function of the lungs.

[0003] The difficulty of three-dimensional reconstruction of pulmonary bronchi or pulmonary arteries and veins is that the computed tomography (CT) images of pulmonary bronchi or pulmonary arteries and veins produce many false positives and faults after three-dimensional reconstruction. Among them, the main reason for the generation of "false positives" is that the segmentation model for segmenting CT images of pulmonary bronchi or pulmonary arteries and veins has poor performance and cannot distinguish image information that is not pulmonary bronchi or pulmonary arteries and veins, thus causing false positives. In addition, "burrs" in pulmonary bronchi or pulmonary arteries and veins are another type of "false positives". If the pulmonary bronchi or pulmonary arteries and veins are regarded as a big tree, then the "burrs" are the countless forked small branches on the trunk of the big tree. The characteristics of these forks are that they are numerous and their lengths are obviously small, which is not very meaningful for analyzing the main trunk of the bronchi or blood vessels, and will also cause visual interference, so they need to be removed. The reason for many faults is that the data for the three-dimensional reconstruction of the pulmonary bronchi or pulmonary arteries and veins is reconstructed from CT images. If the CT image layer thickness is large, data loss is easy, which leads to faults. Second, the pixel threshold of the CT image itself has limitations. Some pulmonary bronchi or pulmonary arteries and veins are not clearly distinguished in the image. If the performance of the segmentation model is not high, some parts of the scan data will be missed, resulting in a lack of overall continuity and faults.

[0004] Because the existing three-dimensional reconstruction technology of the pulmonary bronchi or pulmonary arteries and veins in human lung CT images is poor, there are many faults and false positive problems, and it is impossible to perform accurate three-dimensional reconstruction of the human lung bronchi or pulmonary arteries and veins, which can easily cause missed detection and misdetection. Summary of the invention

[0005] The present disclosure provides a method, device, electronic device and storage medium for optimizing 3D reconstruction of lung images. The main purpose is to solve the problems that the existing 3D reconstruction technology of pulmonary bronchi or pulmonary arteries and veins in human lung CT images has poor effect, many faults and false positives, and cannot accurately reconstruct 3D of human lung bronchi or pulmonary arteries and veins, which easily leads to missed detection and misdetection.

[0006] According to a first aspect of the present disclosure, a method for optimizing three-dimensional reconstruction of lung images is provided, comprising:

[0007] The trained fracture repair model is used to perform vascular repair processing on the target area image in the lung image to obtain a repaired target area image;

[0008] After extracting the first blood vessel skeleton line in the restored target area image, determining the number of adjacent nodes of each node in the first blood vessel skeleton line;

[0009] Determine, according to the number of adjacent nodes of each node, a first type of node and a second type of node in the first blood vessel skeleton line, wherein the first type of node is a node having only one adjacent node, and the second type of node is a node having at least three adjacent nodes;

[0010] Determine the blood vessel skeleton line segments between adjacent first-type nodes and second-type nodes as blood vessel branches;

[0011] Calculating the branch length corresponding to the blood vessel branch; and determining the blood vessel branch whose branch length is less than a preset length threshold as a target branch;

[0012] The target branch is masked to obtain an optimized target region image.

[0013] Optionally, before using the trained fracture repair model to perform blood vessel repair processing on the target area image, the method further includes:

[0014] Calling a centerline extraction algorithm to extract a second blood vessel skeleton line in a training image, wherein the training image includes an original label;

[0015] Randomly marking pixel points on the second blood vessel skeleton line to obtain marked pixel points;

[0016] Performing a morphological dilation operation on the marked pixel points to obtain morphologically dilated pixel points;

[0017] Deleting the pixel points after the morphological expansion to obtain a fracture mask corresponding to the training image;

[0018] The fracture repair model is trained based on the fracture mask and the original label to obtain the trained fracture repair model.

[0019] Optionally, the using of the trained fracture repair model to perform vascular repair processing on the target area image in the lung image to obtain a repaired target area image includes:

[0020] Calling a connected domain search algorithm to search for all blood vessel connected domains in the target area image to obtain a blood vessel connected domain group;

[0021] Determine the largest vascular connection domain in the vascular connection domain group as the first vascular body;

[0022] respectively calculating first distances between other connected domains in the connected domain group except the blood vessel main body and the first blood vessel main body;

[0023] All other connected domains and the blood vessel body are sequentially input into the fracture repair model according to the order of magnitude of the first distances to perform blood vessel repair processing, so as to obtain the repaired target area image.

[0024] Optionally, determining the number of adjacent nodes of each node in the first blood vessel skeleton line includes:

[0025] Taking each node in the first blood vessel skeleton line as the center, determining the number of target adjacent pixel points of the node, wherein the target adjacent pixel points are adjacent pixel points located on the first blood vessel skeleton line;

[0026] The target number of adjacent pixels of the pixel is determined as the number of adjacent nodes of each node.

[0027] Optionally, calculating the branch length corresponding to the blood vessel branch includes:

[0028] Determining the number of pixels constituting the blood vessel branch;

[0029] The number of pixels is determined as the branch length corresponding to the blood vessel branch.

[0030] Optionally, before using the trained fracture repair model to perform vascular repair processing on the target area image in the lung image, the method further includes:

[0031] According to the characteristics of the pulmonary blood vessels in the pulmonary image, the pulmonary image is partitioned to obtain a first region image, a second region image and the target region image;

[0032] After obtaining the optimized target area image, the method further includes:

[0033] Performing a first median filtering process on the first region image to obtain a filtered first region image;

[0034] Performing a second median filtering process on the second region image to obtain a filtered second region image;

[0035] Performing a third median filtering process on the optimized target area image to obtain a filtered target area image; wherein the filter kernel sizes of the first median filtering process, the second median filtering process and the third median filtering process are different;

[0036] The filtered first region image, the filtered second region image and the filtered target region image are merged to obtain a target image.

[0037] According to a second aspect of the present disclosure, a lung image three-dimensional reconstruction optimization device is provided, comprising:

[0038] A repair unit, used for performing blood vessel repair processing on a target area image in a lung image by using a trained fracture repair model to obtain a repaired target area image;

[0039] A first determining unit is used to determine the number of adjacent nodes of each node in the first blood vessel skeleton line after extracting the first blood vessel skeleton line in the restored target area image;

[0040] A second determining unit is used to determine a first type of node and a second type of node in the first blood vessel skeleton line according to the number of adjacent nodes of each node, wherein the first type of node is a node having only one adjacent node, and the second type of node is a node having at least three adjacent nodes;

[0041] A third determining unit is used to determine the blood vessel skeleton line segments between adjacent first-type nodes and second-type nodes as blood vessel branches;

[0042] A calculation unit, used to calculate the branch length corresponding to the blood vessel branch;

[0043] a fourth determining unit, configured to determine the blood vessel branch whose branch length is less than a preset length threshold as a target branch;

[0044] The masking unit is used to perform masking processing on the target branch to obtain an optimized target area image.

[0045] Optionally, the device further comprises a training unit, wherein the training unit comprises:

[0046] An extraction module, used to call a centerline extraction algorithm to extract a second blood vessel skeleton line in a training image before using the trained fracture repair model to perform blood vessel repair processing on the target area image, wherein the training image contains an original label;

[0047] A marking module, used for randomly marking pixel points on the second blood vessel skeleton line to obtain marked pixel points;

[0048] A dilation module, used for performing a morphological dilation operation on the marked pixel points to obtain morphologically dilated pixel points;

[0049] A deletion module, used for deleting the pixels after the morphological expansion to obtain a fracture mask corresponding to the training image;

[0050] A training module is used to train the fracture repair model based on the fracture mask and the original label to obtain the trained fracture repair model.

[0051] Optionally, the repair unit comprises:

[0052] A search module, used for calling a connected domain search algorithm to search for all blood vessel connected domains in the target area image to obtain a blood vessel connected domain group;

[0053] A first determination module, configured to determine the largest vascular connection domain in the vascular connection domain group as a first vascular body;

[0054] A first calculation module, used for respectively calculating first distances between other connected domains in the connected domain group except the blood vessel body and the first blood vessel body;

[0055] The repair module is used to input all other connected domains and the blood vessel body into the fracture repair model in order of the first distance to perform blood vessel repair processing and obtain the repaired target area image.

[0056] Optionally, the second determining unit includes:

[0057] A second determination module is used to determine the number of target adjacent pixel points of each node in the first blood vessel skeleton line, wherein the target adjacent pixel points are adjacent pixel points located on the first blood vessel skeleton line;

[0058] The third determination module is used to determine the target number of adjacent pixels of the pixel as the number of adjacent nodes of each node.

[0059] Optionally, the computing unit includes:

[0060] A fourth determination module, used to determine the number of pixels constituting the blood vessel branch;

[0061] The fifth determining module is used to determine the number of pixel points as the branch length corresponding to the blood vessel branch.

[0062] Optionally, the device further comprises a partition processing and filtering unit, wherein the partition unit is used to:

[0063] Before using the trained fracture repair model to perform vascular repair processing on the target region image in the lung image, the lung image is partitioned according to the characteristics of the lung blood vessels in the lung image to obtain a first region image, a second region image and the target region image;

[0064] The filtering unit comprises:

[0065] A first filtering module, configured to perform a first median filtering process on the first region image after obtaining the optimized target region image, so as to obtain a filtered first region image;

[0066] A second filtering module, used for performing a second median filtering process on the second region image to obtain a filtered second region image;

[0067] A third filtering module is used to perform a third median filtering process on the optimized target area image to obtain a filtered target area image; wherein the filter kernel sizes of the first median filtering process, the second median filtering process and the third median filtering process are different;

[0068] The merging module is used to merge the filtered first region image, the filtered second region image and the filtered target region image to obtain a target image.

[0069] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0070] at least one processor; and

[0071] a memory communicatively connected to the at least one processor; wherein,

[0072] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect.

[0073] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method described in the first aspect.

[0074] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the method as described in the first aspect above.

[0075] The present disclosure provides a lung image three-dimensional reconstruction optimization method, device, electronic device and storage medium, which use a trained fracture repair model to perform vascular repair processing on a target area image in a lung image to obtain a repaired target area image; after extracting a first vascular skeleton line in the repaired target area image, determine the number of adjacent nodes of each node in the first vascular skeleton line; determine a first type of node and a second type of node in the first vascular skeleton line according to the number of adjacent nodes of each node, the first type of node is a node with only one adjacent node, and the second type of node is a node with at least three adjacent nodes; determine the vascular skeleton line segment between adjacent first type nodes and second type nodes as a vascular branch; calculate the branch length corresponding to the vascular branch; and determine the vascular branch whose branch length is less than a preset length threshold as a target branch; mask the target branch to obtain an optimized target area image. Compared with the related art, the lung image three-dimensional reconstruction optimization method provided by the present invention repairs the vascular faults appearing in the lung image, and masks the vascular branches in the repaired lung image whose branch lengths are less than a preset length threshold, thereby reducing the possibility of false positives, solving the fault and false positive problems appearing in the lung image, and achieving the optimization of the lung image.

[0076] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0077] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0078] Figure 1 A schematic diagram of a process flow of a lung image three-dimensional reconstruction optimization method provided by an embodiment of the present disclosure;

[0079] Figure 2 An example of a blood vessel section provided by an embodiment of the present disclosure;

[0080] Figure 3 An example diagram of a blood vessel "burr" provided in an embodiment of the present disclosure;

[0081] Figure 4 A schematic diagram of another process for optimizing three-dimensional reconstruction of lung images provided in an embodiment of the present disclosure;

[0082] Figure 5An example diagram of blood vessels in a lung image provided by an embodiment of the present disclosure;

[0083] Figure 6 A schematic diagram of another process for optimizing three-dimensional reconstruction of lung images provided in an embodiment of the present disclosure;

[0084] Figure 7 A schematic diagram of another process for optimizing three-dimensional reconstruction of lung images provided in an embodiment of the present disclosure;

[0085] Figure 8 A schematic diagram of a lung image three-dimensional reconstruction optimization device provided in an embodiment of the present disclosure

[0086] Fig. 9 A schematic diagram of the structure of another lung image three-dimensional reconstruction optimization device provided by an embodiment of the present disclosure;

[0087] Fig.10 A schematic block diagram of an exemplary electronic device 600 provided for an embodiment of the present disclosure. DETAILED DESCRIPTION

[0088] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0089] In the disclosed embodiments, the word "exemplary" is used to mean "used as an example, illustration or description". Any embodiment described as "exemplary" in the disclosed embodiments is not necessarily to be interpreted as being more preferred or more advantageous than other embodiments. The prefixes such as "first" and "second" in the disclosed embodiments are only used to distinguish different description objects, and do not constitute restrictions on the position, order, priority, quantity or content of the description objects. The statement of the description objects refers to the description in the context of the claims or embodiments, and should not constitute unnecessary restrictions due to the use of prefixes. It should be understood that a person of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other examples, well-known structures and processes will not be elaborated in detail to avoid unnecessary details that make the description of the present invention obscure. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest range of principles and features disclosed in this application.

[0090] It should be noted that since the method of the embodiment of the present application is executed in a computing device, the processing objects of each computing device exist in the form of data or information. For example, time is actually time information. It can be understood that if size, quantity, position, etc. are mentioned in subsequent embodiments, they are all corresponding data for processing by the electronic device. The details will not be elaborated here.

[0091] The following describes the lung image three-dimensional reconstruction optimization method and device, electronic device and storage medium according to the embodiments of the present disclosure with reference to the accompanying drawings.

[0092] Figure 1 A schematic diagram of a flow chart of a lung image three-dimensional reconstruction optimization method provided by an embodiment of the present disclosure, wherein the method is applied in a computer, such as Figure 1 As shown, the method comprises the following steps:

[0093] Step 101 , using the trained fracture repair model to perform vascular repair processing on the target area image in the lung image to obtain a repaired target area image.

[0094] In one embodiment provided by the present disclosure, the lung image is a lung image obtained by performing preliminary segmentation of human pulmonary arteries and veins using a segmentation model and a tomography (CT) image, and the target area image is an image of an area in the lung image containing a large number of pulmonary capillaries, such as Figure 5 As shown in the figure, the characteristics of pulmonary capillaries are dense and disordered, with a large number of vascular branches and small blood vessel diameters. Therefore, the image of the target area is prone to vascular segmentation problems and false positive problems caused by the existence of short vascular branches (i.e. "burrs"). For a better understanding of the vascular segmentation problem and vascular "burrs", please refer to Figure 2 and Figure 3 , Figure 2 An example diagram of a blood vessel section provided in an embodiment of the present disclosure. Figure 3 An example diagram of a blood vessel "burr" provided in an embodiment of the present disclosure.

[0095] Since impurities or new vascular branches are easily introduced during the repair of vascular fractures, thereby causing false positives, in the embodiments provided in the present disclosure, it is necessary to first perform vascular repair processing on the target area image to connect the breakpoints of the vascular fracture, and then perform subsequent processing on the repaired target area image.

[0096] In order to solve the problem of blood vessel fracture in the target area image, the embodiment of the present disclosure provides a trained fracture repair model to implement blood vessel repair processing on the target area image. The trained fracture repair model is used to connect the breakpoints of the blood vessel fractures in the target area image, thereby obtaining the repaired target area image.

[0097] Step 102: after extracting the first blood vessel skeleton line in the restored target region image, determine the number of adjacent nodes of each node in the first blood vessel skeleton line.

[0098] In one embodiment provided by the present disclosure, after obtaining the restored target area image, the first blood vessel skeleton line can be extracted from the restored target area image by using, but not limited to, a skeletonization three-dimensional image algorithm (such as morphology.skeletonize_3d) in an image processing library (such as scikit-image scikit (toolkit for scipy), skimage). The skeleton line is a 4-neighborhood or 8-neighborhood line extracted by a thinning algorithm (binary image skeleton extraction), and the endpoints and intersections of the skeleton are particularly important. Figure 4 As shown, Figure 4 A schematic diagram of a first blood vessel skeleton line provided in an embodiment of the present disclosure, wherein the first blood vessel skeleton line includes a plurality of nodes, and the node refers to a connection point between any two adjacent blood vessel skeleton line segments.

[0099] Step 103, determining first-type nodes and second-type nodes in the first blood vessel skeleton line according to the number of adjacent nodes of each node, wherein the first-type nodes are nodes with only one adjacent node, and the second-type nodes are nodes with at least three adjacent nodes.

[0100] In some embodiments provided by the present disclosure, the types of nodes can be divided according to the number of adjacent nodes of each node. Figure 4 , the first type of node is a node with only one adjacent node, for example Figure 4 The B node in has only one adjacent node A, so the type of the B node is a first-type node, and the second-type node is a node with at least three adjacent nodes. For example, Figure 4 The H node in the diagram has C node, G node, and F node as its neighboring nodes, and the number of its neighboring nodes is 3. Therefore, the H node is the second type of node. In addition, there is a third type of node on the first blood vessel skeleton line. The third type of node refers to a node with two neighboring nodes. For example, Figure 4 The C node in , its adjacent nodes are A node and H node, therefore, the C node is a third type node.

[0101] Step 104: determine the blood vessel skeleton line segments between adjacent first-type nodes and second-type nodes as blood vessel branches.

[0102] In some embodiments provided by the present disclosure, the vascular skeleton line segment refers to all vascular skeletons between any two adjacent nodes in the first vascular skeleton line. Figure 4 , wherein line segment AC, line segment CH, etc. are all blood vessel skeleton line segments. wherein G node and F node are both first-type nodes, and H node is a second-type node, therefore, line segment GH and line segment FH are both blood vessel branches.

[0103] Step 105, calculating the branch length corresponding to the blood vessel branch; and determining the blood vessel branch whose branch length is less than a preset length threshold as a target branch.

[0104] In some embodiments provided in the present disclosure, since the blood vessel branches in the lung image are composed of pixel points, the branch length corresponding to the blood vessel branch can be calculated by counting the number of pixel points between the first type of nodes and the second type of nodes of each blood vessel branch.

[0105] In some embodiments provided in the present disclosure, the preset length threshold can be flexibly adjusted according to actual application conditions. For example, the preset length threshold can be set to 10 pixels, or 15 pixels, etc., which is not limited to the specific embodiments of the present disclosure.

[0106] Step 106: mask the target branch to obtain an optimized target region image.

[0107] The three-dimensional reconstruction optimization method of lung images provided by the present disclosure uses a trained fracture repair model to perform vascular repair processing on the target area image in the lung image to obtain the repaired target area image; after extracting the first vascular skeleton line in the repaired target area image, determine the number of adjacent nodes of each node in the first vascular skeleton line; according to the number of adjacent nodes of each node, determine the first type of node and the second type of node in the first vascular skeleton line, the first type of node is a node with only one adjacent node, and the second type of node is a node with at least three adjacent nodes; determine the vascular skeleton line segments between the adjacent first type of nodes and the second type of nodes as vascular branches; calculate the branch length corresponding to the vascular branch; and determine the vascular branch with a branch length less than a preset length threshold as a target branch; mask the target branch to obtain the optimized target area image. Compared with the related art, the three-dimensional reconstruction optimization method of lung images provided by the present disclosure repairs the vascular faults in the lung image, and masks the vascular branches with a branch length less than a preset length threshold in the repaired lung image, thereby reducing the possibility of false positives, solving the fault and false positive problems in the lung image, and realizing the optimization of the lung image.

[0108] In order to obtain a trained fracture repair model, a method of using a masked auto encoder (MAE) to train a preset fracture repair model may be adopted but is not limited to the method so as to repair the target area image. The embodiment of the present disclosure also provides a flow chart of a fracture repair model training method, as shown in FIG. Figure 6 As shown, the method comprises the following steps:

[0109] Step 201 : calling a centerline extraction algorithm to extract a second blood vessel skeleton line in a training image, wherein the training image contains an original label.

[0110] In one embodiment provided by the present disclosure, in order to simulate the actual situation of blood vessel fracture, the second blood vessel line skeleton of the blood vessel is first extracted using a centerline extraction algorithm. In order to obtain a training data set for training the fracture repair model, the training data set can be prepared using, but not limited to, the Lung Image Database Consortium (LIDC-IDRI) public data set of pulmonary nodules, which consists of chest medical image files (such as CT, X-ray films) and corresponding diagnosis result lesion annotations.

[0111] Step 202: randomly mark the pixel points on the second blood vessel skeleton line to obtain marked pixel points.

[0112] After acquiring the second blood vessel skeleton line, pixels belonging to the capillary portion are randomly marked on the second blood vessel skeleton line.

[0113] Step 203: Perform a morphological dilation operation on the marked pixel points to obtain morphologically dilated pixel points.

[0114] The marked pixels are subjected to a morphological dilation operation. The morphology is also called mathematical morphology, which is a very important research method in the image processing process. Morphology mainly extracts component information from the inside of the image. This component information is usually of great significance for expressing and describing the characteristics of the image, and is usually the most essential shape feature used in image understanding. Morphological processing plays a very important role in visual detection, text recognition, medical graphics processing, and image compression coding. Morphological operations mainly include: erosion, dilation, opening operation, closing operation, morphological gradient operation, top hat operation, and black hat operation. Among them, dilation and erosion are the most basic morphological operation methods in image processing, and are often combined to implement some complex image morphological operations.

[0115] Step 204 , deleting the pixels after the morphological expansion to obtain a fracture mask corresponding to the training image.

[0116] After obtaining the morphologically expanded pixel points, the mask of the corresponding morphologically expanded pixel points on the blood vessel mask is deleted, and the obtained blood vessel mask is a fracture mask after simulating blood vessel fracture.

[0117] Step 205: training the fracture repair model based on the fracture mask and the original label to obtain the trained fracture repair model.

[0118] The fracture mask generated in step 204 and the original label together constitute a training sample. Based on the training sample, the fracture repair model is trained to obtain the trained fracture repair model.

[0119] Through the fracture repair model training method provided in the above embodiment, the fracture repair model is trained to obtain a trained fracture repair model and repair the blood vessels in the target area image.

[0120] In one embodiment provided by the present disclosure, after obtaining the trained fracture repair model, in order to perform vascular repair processing on the target area image in the lung image using the trained fracture repair model to obtain the repaired target area image, a better understanding is provided. Figure 7 A schematic diagram of a fault repair process provided by an embodiment of the present disclosure, such as Figure 7 As shown, the method comprises the following steps:

[0121] Step 301, calling a connected domain search algorithm to search for all blood vessel connected domains in the target region image to obtain a blood vessel connected domain group.

[0122] In one embodiment provided by the present disclosure, the connected region (Connected Component) generally refers to an image region (Region, Blob) composed of foreground pixels with the same pixel value and adjacent positions in the image. Connected domain analysis (Connected Component Analysis, Connected Component Labeling) refers to finding and labeling each connected domain in the image. Connected domain analysis is a relatively common and basic method in many application fields of CVPR and image analysis and processing. For example: character segmentation and extraction (license plate recognition, text recognition, subtitle recognition, etc.) in (Optical Character Recognition, OCR), segmentation and extraction of moving foreground targets in visual tracking (pedestrian intrusion detection, abandoned object detection, vision-based vehicle detection and tracking, etc.), medical image processing (extraction of target regions of interest), etc.

[0123] Step 302: determine the largest vascular connected domain in the vascular connected domain group as the first vascular body.

[0124] After the blood vessel connected domain group is determined, the sizes of all connected domains in the blood vessel connected domain group are sorted, and the largest blood vessel connected domain is determined as the blood vessel body.

[0125] Step 303: Calculate first distances between other connected domains in the connected domain group except the blood vessel body and the first blood vessel body.

[0126] After the blood vessel body is determined, first distances between other connected domains in the connected domain group except the blood vessel body and the first blood vessel body are calculated respectively, and the first distances are sorted.

[0127] Step 304 , according to the order of the first distances, all other connected domains and the blood vessel body are input into the trained fracture repair model in turn to perform blood vessel repair processing, so as to obtain the repaired target area image.

[0128] After obtaining the order of the first distances, the vascular connection domain with the smallest first distance is selected to be combined with the vascular body, and the combination is placed in the trained fracture repair model for repair. The repaired second vascular body is used as the new vascular body, and then the remaining vascular connection domains with the smallest first distance and the second vascular body are input into the trained fracture repair model again until there is only one target vascular body in the vascular connection domain group, thereby obtaining the repaired target area image.

[0129] Through the methods provided in the above steps 301 to 304, the blood vessel fault problem existing in the target area image is repaired.

[0130] As a further explanation of the above step 102, the determination of the number of adjacent nodes of each node in the first vascular skeleton line can be carried out in the following manner but is not limited to: first, with each node in the first vascular skeleton line as the center, the target number of adjacent pixel points of the node is determined, and the target adjacent pixel points are the adjacent pixel points located on the first vascular skeleton line; the target number of adjacent pixel points of the pixel point is determined as the number of adjacent nodes of each node. Exemplarily, with node M as the center, calculate whether the corresponding pixel points at a total of 26 adjacent positions of node M in front, back, top, bottom, left, right, and corresponding oblique directions are on the first vascular skeleton line, and the number of target adjacent pixel points is equal to the number of adjacent nodes.

[0131] After obtaining the branch length of each blood vessel branch, the branch length is compared with a preset length threshold. If the branch length is less than the preset length threshold, the corresponding blood vessel branch is determined to be a "burr". The "burr" may cause a false positive problem, so the "burr" needs to be covered. If the branch length is greater than the preset length threshold, it means that the corresponding blood vessel branch is a normal blood vessel, will not cause a false positive problem, and does not need to be processed.

[0132] Through the method provided in the above embodiment, the false positive problem existing in the repaired target area image is solved, and the three-dimensional reconstruction optimization of the target area image is achieved.

[0133] In one embodiment provided by the present disclosure, before using a trained fracture repair model to perform vascular repair processing on a target area image in a lung image, taking into account the characteristics of the pulmonary blood vessels themselves, the blood vessels are divided into three parts according to the characteristics of the pulmonary blood vessels in the lung image, and each of the three parts is optimized. The lung image is partitioned to obtain a first area image, a second area image and the target area image; the first area image is a vascular image close to the heart, the second area image is a vascular image near the hilum, and the target area image is a pulmonary capillary image.

[0134] First, the blood vessels near the heart. This part can be called the core area of ​​the blood vessels. The blood vessels in the core area of ​​the blood vessels are characterized by a relatively large blood vessel as a whole. The blood vessels in this part have almost no false positives and faults, and there are no bifurcations. The whole is spherical. The main problem that needs to be optimized in this part of the blood vessels is the outer shell of the core area of ​​the blood vessels, which usually presents a step-like shape affected by the thickness of the CT image. Therefore, for this first area image, a median filter with a larger filter kernel can be applied to directly smooth the overall effect.

[0135] As for the blood vessels in the second region image, the blood vessels in this part are the middle area connecting the core area of ​​blood vessels and capillaries. The main features are thick blood vessels and distinct bifurcations. Each bifurcation is connected to a lung lobe. The left lung has two hilar bifurcations, and the right lung has three hilar bifurcations. Since the blood vessels are relatively thick, a median filter with a medium filter kernel is applied to this part.

[0136] For the optimized target area image, median filtering with a smaller filter kernel is directly used for smoothing.

[0137] After respectively performing median filtering processing with different filter kernel sizes on the first area image, the second area image, and the optimized target area image, a filtered first area image, a filtered second area image, and a filtered target area image are obtained; then, the filtered first area image, the filtered second area image, and the filtered target area image are merged to obtain a target image.

[0138] Compared with related technologies, the partitioning method is used to process the blood vessels in the distribution image separately, which can accurately deal with the discontinuity problem and false positive problem in the blood vessels, and improve the performance optimization of the three-dimensional reconstruction technology of the lung image.

[0139] In summary, the embodiments of the present disclosure have the following effects:

[0140] The vascular faults in the lung images were repaired, and the vascular branches in the repaired lung images whose branch lengths were less than the preset length threshold were masked, thereby reducing the possibility of false positives, solving the fault and false positive problems in the lung images, and achieving optimization of the lung images.

[0141] Corresponding to the above-mentioned lung image three-dimensional reconstruction optimization method, the present invention also proposes a lung image three-dimensional reconstruction optimization method device. Since the device embodiment of the present invention corresponds to the above-mentioned method embodiment, the details not disclosed in the device embodiment can be referred to the above-mentioned method embodiment, and will not be repeated in the present invention.

[0142] Figure 8 A schematic diagram of the structure of a lung image three-dimensional reconstruction optimization device provided by an embodiment of the present disclosure, such as Figure 8 As shown, it includes: a repairing unit 41, a first determining unit 42, a second determining unit 43, a third determining unit 44, a calculating unit 45, a fourth determining unit 46, and a concealing unit 47.

[0143] According to a second aspect of the present disclosure, a lung image three-dimensional reconstruction optimization device is provided, comprising:

[0144] A repair unit 41 is used to perform a blood vessel repair process on the target area image in the lung image using a trained fracture repair model to obtain a repaired target area image;

[0145] A first determining unit 42 is used to determine the number of adjacent nodes of each node in the first blood vessel skeleton line after extracting the first blood vessel skeleton line in the restored target area image;

[0146] A second determining unit 43 is used to determine a first type of node and a second type of node in the first blood vessel skeleton line according to the number of adjacent nodes of each node, wherein the first type of node is a node having only one adjacent node, and the second type of node is a node having at least three adjacent nodes;

[0147] A third determining unit 44 is used to determine the blood vessel skeleton line segments between adjacent first-type nodes and second-type nodes as blood vessel branches;

[0148] A calculation unit 45, configured to calculate a branch length corresponding to the blood vessel branch;

[0149] A fourth determining unit 46, configured to determine the blood vessel branch whose branch length is less than a preset length threshold as a target branch;

[0150] The masking unit 47 is used to perform masking processing on the target branch to obtain an optimized target area image.

[0151] The lung image three-dimensional reconstruction optimization device provided by the present disclosure uses a trained fracture repair model to perform vascular repair processing on the target area image in the lung image to obtain the repaired target area image; after extracting the first vascular skeleton line in the repaired target area image, determine the number of adjacent nodes of each node in the first vascular skeleton line; according to the number of adjacent nodes of each node, determine the first type of node and the second type of node in the first vascular skeleton line, the first type of node is a node with only one adjacent node, and the second type of node is a node with at least three adjacent nodes; determine the vascular skeleton line segment between the adjacent first type of node and the second type of node as a vascular branch; calculate the branch length corresponding to the vascular branch; and determine the vascular branch with a branch length less than a preset length threshold as a target branch; mask the target branch to obtain the optimized target area image. Compared with the related art, the lung image three-dimensional reconstruction optimization method provided by the present disclosure repairs the vascular faults in the lung image, and masks the vascular branches with a branch length less than a preset length threshold in the repaired lung image, thereby reducing the possibility of false positives, solving the fault and false positive problems in the lung image, and realizing the optimization of the lung image.

[0152] Furthermore, in a possible implementation of this embodiment, as Fig. 9 As shown, the apparatus further includes a training unit 48, and the training unit 48 includes:

[0153] The extraction module 481 is used to call the centerline extraction algorithm to extract the second blood vessel skeleton line in the training image before using the trained fracture repair model to perform blood vessel repair processing on the target area image, and the training image contains the original label;

[0154] A marking module 482, configured to randomly mark pixel points on the second blood vessel skeleton line to obtain marked pixel points;

[0155] An expansion module 483 is used to perform a morphological expansion operation on the marked pixel points to obtain morphologically expanded pixel points;

[0156] A deletion module 484 is used to delete the pixels after the morphological expansion to obtain a fracture mask corresponding to the training image;

[0157] The training module 485 is used to train the fracture repair model based on the fracture mask and the original label to obtain the trained fracture repair model.

[0158] Optionally, the repair unit 41 includes:

[0159] A search module 411 is used to call a connected domain search algorithm to search for all blood vessel connected domains in the target area image to obtain a blood vessel connected domain group;

[0160] A first determination module 412, configured to determine the largest vascular connected domain in the vascular connected domain group as a first vascular body;

[0161] A first calculation module 413, used for respectively calculating first distances between other connected domains in the connected domain group except the blood vessel body and the first blood vessel body;

[0162] The repair module 414 is used to input all other connected domains and the blood vessel body into the fracture repair model in order of the first distance to perform blood vessel repair processing, so as to obtain the repaired target area image.

[0163] Optionally, the second determining unit 43 includes:

[0164] A second determination module 431 is used to determine the number of target adjacent pixel points of each node in the first blood vessel skeleton line, wherein the target adjacent pixel points are adjacent pixel points located on the first blood vessel skeleton line;

[0165] The third determination module 432 is used to determine the target number of adjacent pixels of the pixel as the number of adjacent nodes of each node.

[0166] Optionally, the calculation unit 45 includes:

[0167] A fourth determination module 451 is used to determine the number of pixels constituting the blood vessel branch;

[0168] The fifth determining module 452 is configured to determine the number of pixels as the branch length corresponding to the blood vessel branch.

[0169] Optionally, the device further includes a partition processing unit 49 and a filtering unit 410, wherein the partition processing unit 45 is used to:

[0170] According to the characteristics of the pulmonary blood vessels in the pulmonary image, the pulmonary image is partitioned to obtain a first region image, a second region image and the target region image;

[0171] The filtering unit 410 includes:

[0172] A first filtering module 4101 is used for performing a first median filtering process on the first region image after obtaining the optimized target region image to obtain a filtered first region image;

[0173] A second filtering module 4102 is used to perform a second median filtering process on the second region image to obtain a filtered second region image;

[0174] A third filtering module 4103 is used to perform a third median filtering process on the optimized target area image to obtain a filtered target area image; wherein the filter kernel sizes of the first median filtering process, the second median filtering process and the third median filtering process are different;

[0175] The merging module 4104 is used to merge the filtered first region image, the filtered second region image and the filtered target region image to obtain a target image.

[0176] It should be noted that the above explanation of the method embodiment is also applicable to the device of this embodiment, and the principle is the same, which is not limited in this embodiment.

[0177] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0178] Fig.10 A schematic block diagram of an example electronic device 500 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0179] like Fig.10 As shown, the device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 502 or a computer program loaded from a storage unit 508 to a RAM (Random Access Memory) 503. Various programs and data required for the operation of the device 500 can also be stored in the RAM 503. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An I / O (Input / Output) interface 505 is also connected to the bus 504.

[0180] A number of components in the device 500 are connected to the I / O interface 505, including: an input unit 506, such as a keyboard, a mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a disk, an optical disk, etc.; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows the device 500 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0181] The computing unit 501 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a CPU (Central Processing Unit), a GPU (Graphic Processing Units), various dedicated AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, a DSP (Digital Signal Processor), and any appropriate processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as a lung image three-dimensional reconstruction optimization method. For example, in some embodiments, the lung image three-dimensional reconstruction optimization method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, one or more steps of the method described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to execute the aforementioned lung image three-dimensional reconstruction optimization method in any other appropriate manner (for example, by means of firmware).

[0182] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application Specific Standard Products), SOCs (System On Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor that may be a dedicated or general-purpose programmable processor that may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0183] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0184] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, RAM, ROM, EPROM (Electrically Programmable Read-Only-Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0185] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0186] The systems and techniques described herein may be implemented in a computing system that includes a backend component (e.g., as a data server), or a computing system that includes a middleware component (e.g., an application server), or a computing system that includes a frontend component (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: LAN (Local Area Network), WAN (Wide Area Network), the Internet, and blockchain networks.

[0187] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0188] It should be noted that artificial intelligence is a discipline that studies how computers can simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.), and includes both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, as well as machine learning / deep learning, big data processing technology, knowledge graph technology, and other major directions.

[0189] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0190] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for optimizing three-dimensional reconstruction of lung images, characterized in that: include: The trained fracture repair model is used to perform vascular repair processing on the target area image in the lung image to obtain a repaired target area image; After extracting the first blood vessel skeleton line in the restored target area image, determining the number of adjacent nodes of each node in the first blood vessel skeleton line; Determine, according to the number of adjacent nodes of each node, a first type of node and a second type of node in the first blood vessel skeleton line, wherein the first type of node is a node having only one adjacent node, and the second type of node is a node having at least three adjacent nodes; Determine the blood vessel skeleton line segments between adjacent first-type nodes and second-type nodes as blood vessel branches; Calculating the branch length corresponding to the blood vessel branch; and determining the blood vessel branch whose branch length is less than a preset length threshold as a target branch; Performing masking processing on the target branch to obtain an optimized target area image; The step of using the trained fracture repair model to perform vascular repair processing on the target region image in the lung image to obtain the repaired target region image includes: calling a connected domain search algorithm to search for all vascular connected domains in the target region image to obtain a vascular connected domain group; Determine the largest vascular connection domain in the vascular connection domain group as the first vascular body; respectively calculating first distances between other connected domains in the connected domain group except the blood vessel main body and the first blood vessel main body; All other connected domains and the blood vessel body are sequentially input into the fracture repair model according to the order of magnitude of the first distances to perform blood vessel repair processing, so as to obtain the repaired target area image.

2. The method according to claim 1, characterized in that Before using the trained fracture repair model to perform blood vessel repair processing on the target area image, the method further includes: Calling a centerline extraction algorithm to extract a second blood vessel skeleton line in a training image, wherein the training image includes an original label; Randomly marking pixel points on the second blood vessel skeleton line to obtain marked pixel points; Performing a morphological dilation operation on the marked pixel points to obtain morphologically dilated pixel points; Deleting the pixel points after the morphological expansion to obtain a fracture mask corresponding to the training image; The fracture repair model is trained based on the fracture mask and the original label to obtain the trained fracture repair model.

3. The method according to claim 1, characterized in that The determining the number of adjacent nodes of each node in the first blood vessel skeleton line includes: Taking each node in the first blood vessel skeleton line as the center, determining the number of target adjacent pixel points of the node, wherein the target adjacent pixel points are adjacent pixel points located on the first blood vessel skeleton line; The target number of adjacent pixels of the pixel is determined as the number of adjacent nodes of each node.

4. The method according to claim 1, characterized in that: The calculating the branch length corresponding to the blood vessel branch includes: Determining the number of pixels constituting the blood vessel branch; The number of pixels is determined as the branch length corresponding to the blood vessel branch.

5. The method according to claim 1, characterized in that Before using the trained fracture repair model to perform vascular repair processing on the target area image in the lung image, the method further includes: According to the characteristics of the pulmonary blood vessels in the pulmonary image, the pulmonary image is partitioned to obtain a first region image, a second region image and the target region image; After obtaining the optimized target area image, the method further includes: Performing a first median filtering process on the first region image to obtain a filtered first region image; Performing a second median filtering process on the second region image to obtain a filtered second region image; Performing a third median filtering process on the optimized target area image to obtain a filtered target area image; wherein the filter kernel sizes of the first median filtering process, the second median filtering process and the third median filtering process are different; The filtered first region image, the filtered second region image and the filtered target region image are merged to obtain a target image.

6. A lung image three-dimensional reconstruction optimization device, characterized in that: include: A repair unit, used for performing blood vessel repair processing on a target area image in a lung image by using a trained fracture repair model to obtain a repaired target area image; A first determining unit is used to determine the number of adjacent nodes of each node in the first blood vessel skeleton line after extracting the first blood vessel skeleton line in the restored target area image; A second determining unit is used to determine a first type of node and a second type of node in the first blood vessel skeleton line according to the number of adjacent nodes of each node, wherein the first type of node is a node having only one adjacent node, and the second type of node is a node having at least three adjacent nodes; A third determining unit is used to determine the blood vessel skeleton line segments between adjacent first-type nodes and second-type nodes as blood vessel branches; A calculation unit, used to calculate the branch length corresponding to the blood vessel branch; a fourth determining unit, configured to determine the blood vessel branch whose branch length is less than a preset length threshold as a target branch; A masking unit, used for performing masking processing on the target branch to obtain an optimized target area image; The repair unit includes: a search module, which is used to call a connected domain search algorithm to search for all blood vessel connected domains in the target area image to obtain a blood vessel connected domain group; A first determination module, configured to determine the largest vascular connection domain in the vascular connection domain group as a first vascular body; A first calculation module, used for respectively calculating first distances between other connected domains in the connected domain group except the blood vessel body and the first blood vessel body; The repair module is used to input all other connected domains and the blood vessel body into the fracture repair model in order of the first distance to perform blood vessel repair processing and obtain the repaired target area image.

7. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-5.

9. A computer program product, characterized in that The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 5.

Images