Image processing method, device and storage medium

Abstract

The present invention discloses an image processing method, device and storage medium. The method comprises: first, extracting a liver region image and a blood vessel region image respectively; then, extracting the blood vessel centerline and dividing the blood vessel centerline into several regions of appropriate size according to the connectivity; then, determining the centerline region corresponding to the portal vein from several blood vessel centerline regions according to the positional relationship between the portal vein and the liver region and the structural characteristics of the portal vein; then, performing diameter recovery according to the centerline region corresponding to the portal vein to obtain a portal vein image, so as to distinguish it from the hepatic vein image. In this way, the portal vein and the hepatic vein can be quickly and accurately distinguished from the blood vessel region.

Description

Technical Field

[0001] The present invention relates to the field of computer image processing, and in particular to an image processing method, device and storage medium. Background Art

[0002] With the rapid development of computer-aided diagnosis technology and computer image processing technology, medical-aided diagnosis systems that extract features from large amounts of medical imaging data (such as MRI and CT) for machine learning and combine them with medical knowledge are gaining more and more attention.

[0003] In particular, the liver auxiliary diagnosis system can segment the areas of interest to doctors from medical images, such as liver areas, tumor areas, blood vessel areas, etc., by using computer image processing, and perform feature extraction and three-dimensional reconstruction on these areas, and classify the segmented blood vessels, which can better help doctors understand the anatomical structure of the liver and make corresponding diagnoses and treatments. The liver vascular system mainly includes the hepatic vein and portal vein systems. Therefore, when distinguishing the liver blood vessels, they need to be divided into the hepatic vein area and the portal vein area. The existing method mainly distinguishes the hepatic vein area and the portal vein area based on the difference in anatomical structure, but this method has a large processing volume and is difficult to adapt to the real-time liver auxiliary diagnosis system.

[0004] How to optimize the existing solutions, improve processing efficiency and shorten calculation time is a technical problem that needs to be solved urgently to further promote the liver auxiliary diagnosis system. Summary of the invention

[0005] In view of the above problems, embodiments of the present invention provide an image processing method, device and storage medium.

[0006] According to a first aspect of an embodiment of the present invention, an image processing method is applied to a medical auxiliary diagnosis system, the method comprising: acquiring first image data corresponding to a liver region and second image data corresponding to a blood vessel region, wherein the blood vessels in the blood vessel region include a hepatic vein and a portal vein; obtaining third image data corresponding to the centerline of the blood vessel according to the second image data and a centerline calculation method, wherein the third image data includes pixel points; performing connected domain division on the third image data according to a connected domain algorithm and a first rule to obtain at least two subsets; determining a portal vein centerline subset corresponding to the portal vein centerline from the at least two subsets according to the first image data and the second rule, wherein the second rule includes a rule for determining the portal vein centerline subset according to a positional relationship between the portal vein and the liver region and a structural feature of the portal vein; determining the image data corresponding to the portal vein from the second image data according to the portal vein centerline subset, so as to distinguish the second image data into image data corresponding to the portal vein and image data corresponding to the hepatic vein.

[0007] According to an embodiment of the present invention, obtaining first image data corresponding to a liver region and second image data corresponding to a blood vessel region includes: obtaining original image data, wherein the original image data includes image data corresponding to the liver region and image data corresponding to the blood vessel region; inputting the original image data into an image processing model to obtain first image data corresponding to the liver region and second image data corresponding to the blood vessel region, wherein the image processing model is an image processing model obtained by training based on a large amount of training image data based on a deep learning method.

[0008] According to an implementation manner of an embodiment of the present invention, the first rule includes: the number of pixel points in the subset is greater than the first threshold and less than the second threshold; accordingly, the third image data is divided into at least two subsets by a connected domain algorithm and the first rule, including: the third image data is divided into at least two connected domain subsets by a connected domain algorithm; it is detected whether the number of pixel points in each connected domain subset is less than the first threshold, if so, the corresponding connected domain subset is merged into any adjacent connected domain subset to obtain a subset that meets the first rule, if not, it is further detected whether the number of pixel points in each connected domain subset is greater than the second threshold, if so, the corresponding connected domain subset is divided into at least two subsets that meet the first rule, if not, the connected domain subset is retained.

[0009] According to an implementation of an embodiment of the present invention, a portal vein centerline subset corresponding to the portal vein centerline is determined from at least two subsets according to the first image data and the second rule, including: determining at least one first subset corresponding to the centerline of the blood vessels outside the liver area according to the first image data and the positional relationship between the pixels, and determining at least one first subset as the subset corresponding to the portal vein centerline; calculating the distance between each first subset and the first image data and determining the first subset with the smallest distance to the first image data as the subset corresponding to the root of the portal vein; determining other subsets of the portal vein centerline subset corresponding to the portal vein centerline according to the subset corresponding to the root of the portal vein and the rule for determining the portal vein centerline subset according to the structural characteristics of the portal vein.

[0010] According to an embodiment of the present invention, other subsets in the portal vein centerline subset corresponding to the portal vein centerline are determined based on the subset corresponding to the portal vein root and the rule for determining the portal vein centerline subset based on the structural characteristics of the portal vein, including: setting the subset corresponding to the portal vein root as the root subset; obtaining a subset set adjacent to the root node subset, detecting whether each subset in the subset set is a subset that has been determined to correspond to the portal vein centerline, if so, setting the corresponding subset as a leaf subset, if not, further performing corresponding processing based on the number of adjacent subsets of the corresponding subset, wherein if the number of adjacent subsets is greater than 1, further calculating the angle between the corresponding subset and the root node subset and determining whether to set the corresponding subset as a leaf subset based on the angle, and if the number of adjacent subsets is equal to 1, setting the corresponding subset as a leaf subset; detecting whether the number of leaf subsets is 0, if so, terminating this execution, if not, setting the leaf subsets as root subsets in turn.

[0011] According to an implementation manner of an embodiment of the present invention, calculating the angle between a corresponding subset and a root subset includes: determining adjacent points between the corresponding subset and the root subset; determining a first farther point in the corresponding subset that is relatively farther from the adjacent point based on the adjacent points; constructing a first direction vector corresponding to the corresponding subset based on the adjacent points and the first farther point; determining a second farther point in the root subset that is relatively farther from the adjacent point based on the adjacent points; constructing a second direction vector corresponding to the root subset based on the adjacent points and the second farther point; and calculating the angle between the corresponding subset and the root subset based on the first direction vector and the second direction vector.

[0012] According to an implementation of an embodiment of the present invention, whether to set the corresponding subset as a leaf subset is determined according to the angle, including: if the angle is greater than 90 degrees, setting the corresponding subset as a leaf subset, otherwise not setting the corresponding subset as a leaf subset.

[0013] According to an implementation of an embodiment of the present invention, image data corresponding to the portal vein is determined from the second image data based on the portal vein centerline subset, including performing the following steps for each subset in the portal vein centerline subset: obtaining the maximum point and the minimum point in each subset based on the pixel point position; determining an extended image data corresponding to each subset based on the maximum point and the minimum point; and determining the second image data that intersects with the extended image data as the image data corresponding to the portal vein.

[0014] According to a second aspect of an embodiment of the present invention, an image processing device is applied to a dialogue system, the device comprising: an image data acquisition module, used to acquire first image data corresponding to a liver region and second image data corresponding to a blood vessel region, wherein the blood vessels in the blood vessel region include a hepatic vein and a portal vein; a centerline calculation module, used to obtain third image data corresponding to the centerline of the blood vessel according to the second image data and a centerline calculation method, wherein the third image data includes pixel points; a connected domain division module, used to perform connected domain division on the third image data according to a connected domain algorithm and a first rule to obtain at least two subsets; a portal vein centerline determination module, used to determine a portal vein centerline subset corresponding to the portal vein centerline from at least two subsets according to the first image data and the second rule, wherein the second rule includes a rule for determining the portal vein centerline subset according to a positional relationship between the portal vein and the liver region and a structural feature of the portal vein; a portal vein determination module, used to determine the image data corresponding to the portal vein from the second image data according to the portal vein centerline subset, so as to distinguish the second image data into image data corresponding to the portal vein and image data corresponding to the hepatic vein.

[0015] According to a third aspect of an embodiment of the present invention, a computer storage medium includes a group of computer executable instructions, and when the instructions are executed, they are used to execute any one of the above-mentioned image processing methods.

[0016] The embodiment of the present invention provides an image processing method, device and storage medium. The method includes: first, extracting a liver region image and a blood vessel region image respectively; next, extracting the blood vessel centerline and dividing the blood vessel centerline into several regions of appropriate size according to the connectivity; then, according to the positional relationship between the portal vein and the liver region and the structural characteristics of the portal vein, determining the centerline region corresponding to the portal vein from several blood vessel centerline regions; then, performing centerline recovery according to the centerline region corresponding to the portal vein to obtain a portal vein image to distinguish it from the hepatic vein image. Among them, extracting the centerline can simplify the hepatic vein and portal vein images into a dendritic linear structure, and is conducive to combining the positional relationship between the portal vein and the liver region and the structural characteristics of the portal vein to quickly distinguish the hepatic vein and the portal vein, which greatly improves the real-time performance of the liver tumor auxiliary diagnosis system for real-time analysis.

[0017] It should be understood that the teachings of the present invention are not required to achieve all of the beneficial effects described above, but specific technical solutions can achieve specific technical effects, and other embodiments of the present invention can also achieve beneficial effects not mentioned above. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-limiting manner, in which:

[0019] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts.

[0020] Figure 1 Schematic diagram of the implementation process of the image processing method according to the embodiment of the present invention;

[0021] Figure 2 This is a schematic diagram of a specific implementation flow of an image processing method according to an embodiment of the present invention;

[0022] Figure 3 This is a schematic diagram of a blood vessel region applied to the image processing method 1 according to an embodiment of the present invention;

[0023] Figure 4 This is a schematic diagram of a blood vessel centerline applied to the image processing method 1 according to an embodiment of the present invention;

[0024] Figure 5 This is a schematic diagram of the result of distinguishing the centerline of the portal vein using the image processing method 1 according to an embodiment of the present invention;

[0025] Figure 6 This is a schematic diagram of the portal vein region differentiation result of applying the image processing method 1 according to an embodiment of the present invention;

[0026] Figure 7 Schematic diagram of the composition structure of an image processing device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0028] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.

[0029] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0030] Figure 1 The implementation process of the image processing method of the embodiment of the present invention is shown, and the method is mainly applied to the medical auxiliary diagnosis system. Figure 1 The method includes: operation 110, obtaining first image data corresponding to a liver region and second image data corresponding to a blood vessel region, wherein the blood vessels in the blood vessel region include a hepatic vein and a portal vein; operation 120, obtaining third image data corresponding to the centerline of the blood vessel according to the second image data and a centerline calculation method, wherein the third image data includes pixel points; operation 130, performing connected domain division on the third image data according to a connected domain algorithm and a first rule to obtain at least two subsets; operation 140, determining a portal vein centerline subset corresponding to the portal vein centerline from the at least two subsets according to the first image data and the second rule, wherein the second rule includes a rule for determining the portal vein centerline subset according to a positional relationship between the portal vein and the liver region and a structural feature of the portal vein; operation 150, determining image data corresponding to the portal vein from the second image data according to the portal vein centerline subset, so as to distinguish the second image data into image data corresponding to the portal vein and image data corresponding to the hepatic vein.

[0031] In operation 110, the first image data corresponding to the liver region and the second image data corresponding to the blood vessel region are mainly derived from a medical image sequence, such as an MRI or CT image sequence, and are extracted by a certain image segmentation algorithm. The embodiment of the present invention does not limit the specific data source and image segmentation algorithm, and the implementer can use any applicable data source and image segmentation algorithm. However, the implementation effect of the embodiment of the present invention will be affected by the accuracy of the image segmentation algorithm. The higher the accuracy of the image segmentation algorithm used by the implementer, the higher the accuracy of the image processing method of the embodiment of the present invention in distinguishing the portal vein and the hepatic vein.

[0032] The first image data corresponding to the liver region is mainly used as a reference for distinguishing the portal vein from the hepatic vein; the second image data corresponding to the blood vessel region is image data including the portal vein and the hepatic vein, and is the object that is important to be processed in the embodiment of the present invention.

[0033] In operation 120, the blood vessel centerline refers to a line formed by connecting the center points of each section perpendicular to the blood vessel direction, which can better reflect the blood vessel structure; there are many methods for calculating the centerline, and the embodiment of the present invention does not limit the centerline calculation method. The implementer can use any applicable centerline calculation method, such as a blood vessel centerline extraction algorithm based on a minimum path, a PMPP-BT algorithm based on an active contour model, etc. However, the implementation effect of the embodiment of the present invention will be affected by the accuracy of the centerline calculation method. The higher the accuracy of the centerline algorithm used by the implementer, the higher the accuracy of the image processing method of the embodiment of the present invention in distinguishing the portal vein and the hepatic vein.

[0034] Among them, extracting the blood vessel centerline can simplify the object of image processing and is particularly suitable for distinguishing the portal vein and the hepatic vein by utilizing the difference in blood vessel structural characteristics.

[0035] In operation 130, the connected region generally refers to an image region composed of adjacent foreground pixels in a binary image. Connected domain analysis refers to finding and marking each connected region in the image in the figure. Connected domain partitioning refers to dividing an overly large connected region into connected regions of appropriate size based on the connected domain analysis. The embodiment of the present invention does not limit the connected domain algorithm, and the implementer may use any applicable connected domain algorithm, such as the Two-Pass algorithm, the Seed-Filling algorithm, and the like. However, the implementation effect of the embodiment of the present invention will be affected by the accuracy of the connected domain algorithm. The higher the accuracy of the connected domain algorithm used by the implementer, the higher the accuracy of the image processing method of the embodiment of the present invention in distinguishing the portal vein and the hepatic vein.

[0036] Among them, converting the blood vessel centerlines into connected areas of appropriate size is helpful for regional processing, which can simplify the complexity and amount of calculations used to determine whether the current centerline subset is the portal vein centerline subset each time, and correspondingly improve the processing accuracy and judgment accuracy.

[0037] In operation 140, the second rule is used to distinguish the centerline of the portal vein from the centerline of the hepatic vein, and to determine the centerline of the portal vein. The inventors have found that the centerline of the portal vein and the centerline of the hepatic vein can be distinguished relatively easily and accurately by combining the positional relationship between the portal vein and the liver region with the unique structural features of the portal vein. The implementer can also add other rules on this basis to adjust or optimize the final result.

[0038] Through this step, the centerline subset of appropriate size can be divided into two parts, one part is the portal vein centerline subset corresponding to the portal vein centerline, and the other part is the hepatic vein centerline subset corresponding to the hepatic vein centerline.

[0039] In operation 150, based on the portal vein centerline subset and the hepatic vein centerline subset obtained in operation 140, image data corresponding to the portal vein and image data corresponding to the hepatic vein are respectively determined from the second image data (image data corresponding to the vascular region). In this way, the hepatic vein and the portal vein are distinguished.

[0040] The image processing method of the embodiment of the present invention simplifies the images of the hepatic vein and the portal vein into a tree-like linear structure by extracting the center line, and then combines the positional relationship between the portal vein and the liver area and the structural characteristics of the portal vein to quickly distinguish the center line of the hepatic vein and the center line of the portal vein. Afterwards, the vascular area of ​​the hepatic vein and the vascular area corresponding to the portal vein are determined based on the center line of the hepatic vein and the center line of the portal vein, thereby realizing a method for quickly distinguishing the hepatic vein and the portal vein, and correspondingly improving the real-time performance of the real-time analysis liver tumor auxiliary diagnosis system.

[0041] According to an embodiment of the present invention, obtaining first image data corresponding to a liver region and second image data corresponding to a blood vessel region includes: obtaining original image data, wherein the original image data includes image data corresponding to the liver region and image data corresponding to the blood vessel region; inputting the original image data into an image processing model to obtain first image data corresponding to the liver region and second image data corresponding to the blood vessel region, wherein the image processing model is an image processing model obtained by training based on a large amount of training image data based on a deep learning method.

[0042] In this embodiment, an image processing model trained with a large amount of training image data based on a deep learning method is used to extract image data corresponding to the liver area and image data corresponding to the blood vessel area from the original image data, which is more accurate and precise, thereby being able to more accurately distinguish between the hepatic vein and the portal vein.

[0043] According to an implementation manner of an embodiment of the present invention, the first rule includes: the number of pixel points in the subset is greater than the first threshold and less than the second threshold; accordingly, the third image data is divided into at least two subsets by a connected domain algorithm and the first rule, including: the third image data is divided into at least two connected domain subsets by a connected domain algorithm; it is detected whether the number of pixel points in each connected domain subset is less than the first threshold, if so, the corresponding connected domain subset is merged into any adjacent connected domain subset to obtain a subset that meets the first rule, if not, it is further detected whether the number of pixel points in each connected domain subset is greater than the second threshold, if so, the corresponding connected domain subset is divided into at least two subsets that meet the first rule, if not, the connected domain subset is retained.

[0044] As mentioned above, the division of connected domain subsets is mainly for regional processing to simplify the computational complexity and amount of each operation used to determine whether the current centerline subset is the portal vein centerline subset. However, it should be noted that if the size of each connected domain subset is too large, the purpose of simplifying the computational complexity cannot be achieved; conversely, if the size of each connected domain subset is too large, the number of executions of the corresponding operation will increase.

[0045] Therefore, in this embodiment, the first rule mainly controls the size of the connected domain subset by controlling the number of pixels so that it can be controlled within a moderate size range. The initial values ​​of the first threshold and the second threshold can be specified based on expert experience or domain knowledge, and can be adjusted based on the implementation effect to achieve better implementation effect.

[0046] According to an implementation of an embodiment of the present invention, a portal vein centerline subset corresponding to the portal vein centerline is determined from at least two subsets according to the first image data and the second rule, including: determining at least one first subset corresponding to the centerline of the blood vessels outside the liver area according to the first image data and the positional relationship between the pixels, and determining at least one first subset as the subset corresponding to the portal vein centerline; calculating the distance between each first subset and the first image data and determining the first subset with the smallest distance to the first image data as the subset corresponding to the root of the portal vein; determining other subsets of the portal vein centerline subset corresponding to the portal vein centerline according to the subset corresponding to the root of the portal vein and the rule for determining the portal vein centerline subset according to the structural characteristics of the portal vein.

[0047] The inventors found that among the hepatic vein and the portal vein, only the root of the portal vein is outside the liver region. Therefore, in the embodiment of the present invention, the inventors cleverly utilized this feature and proposed the following method for distinguishing the hepatic vein centerline and the portal vein centerline: firstly, a subset of vascular centerlines outside the liver region is determined, and then the subset closest to the lower end of the liver is selected as the portal vein root subset, and the portal vein centerline subset is determined based on this subset as the starting point.

[0048] This method is easier to perform than distinguishing only based on the structural characteristics of the main trunks of the portal vein and hepatic vein, and can follow the clues to obtain more blood vessels at the branches, making the distinction more precise and accurate.

[0049] According to an embodiment of the present invention, other subsets in the portal vein centerline subset corresponding to the portal vein centerline are determined based on the subset corresponding to the portal vein root and the rule for determining the portal vein centerline subset based on the structural characteristics of the portal vein, including: setting the subset corresponding to the portal vein root as the root subset; obtaining a subset set adjacent to the root node subset, detecting whether each subset in the subset set is a subset that has been determined to correspond to the portal vein centerline, if so, setting the corresponding subset as a leaf subset, if not, further performing corresponding processing based on the number of adjacent subsets of the corresponding subset, wherein if the number of adjacent subsets is greater than 1, further calculating the angle between the corresponding subset and the root node subset and determining whether to set the corresponding subset as a leaf subset based on the angle, and if the number of adjacent subsets is equal to 1, setting the corresponding subset as a leaf subset; detecting whether the number of leaf subsets is 0, if so, terminating this execution, if not, setting the leaf subsets as root subsets in turn.

[0050] In this embodiment, the inventors found that the growth direction of the portal vein is basically towards the edge of the liver. Therefore, this structural feature is cleverly used to determine whether the adjacent subset of the root subset is the subset corresponding to the center line of the portal vein, mainly by calculating the angle between the root subset and the adjacent subset. In this way, it is relatively easy and accurate to determine whether the adjacent subset of the root subset is the subset corresponding to the center line of the portal vein, and it is not easy to miss suspicious blood vessel branches, so as to achieve full coverage.

[0051] According to an implementation manner of an embodiment of the present invention, calculating the angle between a corresponding subset and a root subset includes: determining adjacent points between the corresponding subset and the root subset; determining a first farther point in the corresponding subset that is relatively farther from the adjacent point based on the adjacent points; constructing a first direction vector corresponding to the corresponding subset based on the adjacent points and the first farther point; determining a second farther point in the root subset that is relatively farther from the adjacent point based on the adjacent points; constructing a second direction vector corresponding to the root subset based on the adjacent points and the second farther point; and calculating the angle between the corresponding subset and the root subset based on the first direction vector and the second direction vector.

[0052] In this embodiment, the direction vector of the corresponding subset and the root subset is determined according to the adjacent points of the corresponding subset and the root subset, and the angle between the corresponding subset and the root subset is calculated by the direction vector, which has a smaller error and can more accurately distinguish the centerline of the portal vein and the centerline of the hepatic vein.

[0053] According to an implementation of an embodiment of the present invention, whether to set the corresponding subset as a leaf subset is determined according to the angle, including: if the angle is greater than 90 degrees, setting the corresponding subset as a leaf subset, otherwise not setting the corresponding subset as a leaf subset.

[0054] In this embodiment, assuming that the angle is greater than 90 degrees, it is more consistent with the structural feature that the growth direction of the portal vein is basically growing toward the edge of the liver. When using this annotation for judgment, the effect of distinguishing the center line of the portal vein and the center line of the hepatic vein is better.

[0055] According to an implementation of an embodiment of the present invention, image data corresponding to the portal vein is determined from the second image data based on the portal vein centerline subset, including performing the following steps for each subset in the portal vein centerline subset: obtaining the maximum point and the minimum point in each subset based on the pixel point position; determining an extended image data corresponding to each subset based on the maximum point and the minimum point; and determining the second image data that intersects with the extended image data as the image data corresponding to the portal vein.

[0056] In this embodiment, the image data is expanded through the maximum point and the minimum point in the center line subset, so that the intersection range with the image data can be expanded by expanding the image data, and it is easier to determine the image data corresponding to the portal vein.

[0057] Figure 2 The specific implementation process of the image processing method of the embodiment of the present invention is shown as follows: Figure 2 As shown, the application mainly distinguishes the vascular images of the liver region into portal vein vascular images and hepatic vein vascular images through the following steps:

[0058] Step 2010, acquiring liver region image data and blood vessel region image data;

[0059] Specifically, a medical image sequence, such as an MRI or CT image sequence, is obtained, and the liver region in the image is obtained based on a deep learning method such as U-net3D. mask and vascular area V original Among them, the vascular area V original Contains the hepatic and portal veins, such as Figure 3 shown.

[0060] Step 2020, extracting blood vessel centerline image data;

[0061] Specifically, a 3D thinning algorithm is used to find the centerline of the blood vessel from the blood vessel region V original Extract the centerline of the blood vessel Skeleton original ,like Figure 4 shown.

[0062] Step 2030, dividing the blood vessel centerline image data into a plurality of subsets;

[0063] Specifically, the 3D connected domain analysis of 18 neighborhoods is used to convert the vascular centerline Skeleton original The lines are divided into multiple subsets. Then the number of pixels in each subset is detected. If the number of pixels is less than 5, they are merged into the subset corresponding to any surrounding connected domain; if the number of pixels is greater than 100, the subset corresponding to the connected domain is divided into multiple subsets with a maximum of 50 pixels. The specific method is to assume that the coordinates of the pixels in the connected domain are all (x, y, z), select a pixel point i on the edge of the connected domain, such as the point with the minimum x value, the point with the minimum y value, or the point with the minimum z value, and use the 18-neighborhood method to search near the pixel point, and count the nearby pixels into the area where point i is located, and stop when the number of pixels is greater than 50, and perform the same operation on the pixels not counted in area i until the number of pixels in each area is no more than 50. After completing the connected domain analysis and post-processing, the vascular centerline Skeleton original is divided into n subsets s n The final effect is as follows Figure 5 As shown, different subsets are displayed as areas with different shades of ink.

[0064] Step 2040, determining a subset corresponding to the portal vein root;

[0065] Specifically, in the n vessel centerline subset s n First, select the subset corresponding to the root of the portal vein in the liver region Liver mask The m blood vessel centerline subsets s m , and then select the subset closest to the lower end of the liver among the m subsets as the subset corresponding to the root of the portal vein pv root At the same time, these m vascular regions are marked as portal vein regions, and the remaining nm regions are marked as non-portal vein regions.

[0066] Step 2050, obtaining a subset set adjacent to the root node subset, and obtaining the first subset;

[0067] Specifically, the subset pv corresponding to the portal vein root root Set it as the root subset and use it as the starting point to search for pv root The corresponding subset of the nearby area, assuming that there are a total of I subsets d nearby i , get a subset di .

[0068] Step 2060, detect the subset d i Whether it is a subset that has been determined to correspond to the centerline of the portal vein, if so, proceed to step 2070, if not, proceed to step 2080;

[0069] Step 2070, subset d i It is the subset that has been determined to correspond to the centerline of the portal vein, and this subset is set as the leaf subset;

[0070] Step 2080, subset d i If it is not a subset that has been determined to correspond to the centerline of the portal vein, determine whether the subset is a subset corresponding to the portal vein, perform corresponding processing, and then obtain the next subset;

[0071] Specifically, if d i belongs to the m regions that have been marked as portal veins, then d i Note: pv root leaf nodes, otherwise find each region d i Nearby Areas j , a total of J dnear j If J = 1, it means area d i Only with portal vein area pv root If the area d i Note: pv root If J>1, it means region d i There are still PV removal nearby root For the blood vessel area outside the i With pv root The angle between them is that the portal vein grows towards the edge of the liver, so d i With pv root The angle between them needs to be greater than 90 degrees. i With pv root When the angle between is greater than 90 degrees, d i Note: pv root The leaf node of d i Note: pv root The leaf nodes of .

[0072] The specific method for calculating the angle between regions is:

[0073] 1) First calculate d i With pv root Adjacent pixels, which belong to d i The neighboring pixels of Belong to pvroot The neighboring pixels If you calculate pv root in and d i Adjacent pixels can be used to find pv root The 26 neighborhoods of all pixels in the 26 neighborhoods, when there are d i If the point in i With pv root Adjacent pixels

[0074] 2) Calculate d i Zhongli Farther pixels For example, select 5 adjacent pixels The farthest pixel, first calculate d i The average distance between other pixels except adjacent pixels and adjacent pixels. The larger the distance average, the farther the distance.

[0075] 3) Calculate pv root Zhongli Farther pixels

[0076] 4) Then find d i With pv root Angle between:

[0077] First, calculate the area d i Direction vector First, calculate the position mean of adjacent pixels and the position mean of farther points:

[0078]

[0079]

[0080] Then d i Direction vector for:

[0081]

[0082] Then, calculate the area pv root Direction vector

[0083] Then d i With pv root The angle between them is:

[0084]

[0085] When cosα is less than 0, it means the angle between them is greater than 90 degrees.

[0086] Step 2100, checking whether the number of the leaf subset is 0, if not, proceeding to step 2110, if yes, proceeding to step 2120;

[0087] Step 2110, if the number of leaf subsets is not 0, then the leaf subsets are set as root subsets in sequence;

[0088] Specifically, the leaf nodes of the root node are regarded as new root nodes, and step 2050 is repeated until the number of subsets near all leaf nodes is 0, or all of them do not meet the conditions for adding as leaf nodes. root The tree-like linear structure of the portal vein centerline with the root node as the portal vein centerline is constructed and marked as the portal vein centerline subset, such as Figure 5 The lighter areas of ink are shown in .

[0089] Step 2120, determining image data corresponding to the portal vein from the vascular region image data according to the portal vein centerline subset;

[0090] Specifically, the subset marked as the centerline of the portal vein corresponds to the original vessel region V original During the operation, the diameter of the portal vein centerline is restored. The restoration method is:

[0091] Find the subset of vessel centerlines labeled portal vein n Location: First, in s n The maximum value x of the pixel point in the x, y, and z coordinates max ,y max , z max and the minimum value x min ,y min , z min ; Then, in the vascular region V original Get an expanded area, such as expanding 30 pixels outward, that is, select V original [x min -15:x max +15,y min -15:y max +15, z min -15:z max +15] area, and other blood vessel areas are set to zero, and the center line s is obtained. n Nearby blood vessels V cut , then V cut Center and centerline subsets n The intersecting vascular connection area is set as the portal vein. The final recovery effect is as follows Figure 6 As shown, the blood vessels with lighter ink are the portal veins.

[0092] It should be noted that the specific implementation process of the image processing method of the above embodiment is only an exemplary description and is not a limitation on the specific implementation process of the image processing method of the embodiment. The implementer can select any applicable implementation method or specific process according to the implementation conditions.

[0093] Furthermore, an embodiment of the present invention also provides an image processing device, which is applied to a dialogue system, such as Figure 7 As shown, the device 70 includes: an image data acquisition module 701, which is used to acquire first image data corresponding to a liver region and second image data corresponding to a blood vessel region, wherein the blood vessels in the blood vessel region include a hepatic vein and a portal vein; a centerline calculation module 702, which is used to obtain third image data corresponding to the centerline of the blood vessel according to the second image data and the centerline calculation method, wherein the third image data includes pixel points; a connected domain division module 703, which is used to perform connected domain division on the third image data according to a connected domain algorithm and a first rule to obtain at least two subsets; a portal vein centerline determination module 704, which is used to determine a portal vein centerline subset corresponding to the portal vein centerline from at least two subsets according to the first image data and the second rule, wherein the second rule includes a rule for determining the portal vein centerline subset according to the positional relationship between the portal vein and the liver region and the structural characteristics of the portal vein; a portal vein determination module 705, which is used to determine the image data corresponding to the portal vein from the second image data according to the portal vein centerline subset, so as to distinguish the second image data into image data corresponding to the portal vein and image data corresponding to the hepatic vein.

[0094] According to an embodiment of the present invention, the image data acquisition module 701 includes: an original image data acquisition submodule, used to acquire original image data, wherein the original image data includes image data corresponding to the liver region and image data corresponding to the blood vessel region; an image processing submodule, used to input the original image data into an image processing model to obtain first image data corresponding to the liver region and second image data corresponding to the blood vessel region, wherein the image processing model is an image processing model obtained by training with a large amount of training image data based on a deep learning method.

[0095] According to an implementation manner of an embodiment of the present invention, the first rule includes: the number of pixel points in the subset is greater than the first threshold and less than the second threshold; accordingly, the connected domain partitioning module 702 includes: a connected domain subset partitioning submodule, used to perform connected domain partitioning on the third image data according to a connected domain algorithm to obtain at least two connected domain subsets; a connected domain subset processing submodule, used to detect whether the number of pixel points in each connected domain subset is less than the first threshold, if so, merge the corresponding connected domain subset into any adjacent connected domain subset to obtain a subset that meets the first rule, if not, further detect whether the number of pixel points in each connected domain subset is greater than the second threshold, if so, divide the corresponding connected domain subset into at least two subsets that meet the first rule, if not, retain the connected domain subset.

[0096] According to an embodiment of the present invention, the portal vein centerline determination module 704 includes: a portal vein centerline first subset determination submodule, which is used to determine at least one first subset corresponding to the blood vessel centerline outside the liver area according to the first image data and the positional relationship between the pixel points, and determine at least one first subset as the subset corresponding to the portal vein centerline; a portal vein root determination submodule, which is used to calculate the distance between each first subset and the first image data and determine the first subset with the smallest distance to the first image data as the subset corresponding to the portal vein root; a portal vein centerline other subset determination submodule, which is used to determine other subsets of the portal vein centerline subset corresponding to the portal vein centerline according to the subset corresponding to the portal vein root and the rule for determining the portal vein centerline subset according to the structural characteristics of the portal vein.

[0097] According to an embodiment of the present invention, other subset determination submodules of the portal vein centerline include: a root subset determination unit, used to set the subset corresponding to the root of the portal vein as the root subset; a leaf subset determination unit, used to obtain a subset set adjacent to the root node subset, and detect whether each subset in the subset set is a subset that has been determined to correspond to the portal vein centerline. If so, the corresponding subset is set as a leaf subset; if not, further corresponding processing is performed according to the number of adjacent subsets of the corresponding subset, wherein if the number of adjacent subsets is greater than 1, the angle between the corresponding subset and the root node subset is further calculated and whether the corresponding subset is set as a leaf subset is determined according to the angle; if the number of adjacent subsets is equal to 1, the corresponding subset is set as a leaf subset; a leaf subset detection unit, used to detect whether the number of leaf subsets is 0, if so, terminate this execution, if not, set the leaf subsets as root subsets in turn.

[0098] According to an implementation manner of an embodiment of the present invention, a leaf subset determination unit includes: an adjacent point determination subunit, used to determine adjacent points between a corresponding subset and a root subset; a first farther point determination subunit, used to determine a first farther point in a corresponding subset that is relatively farther from the adjacent points based on the adjacent points; a first direction vector construction subunit, used to construct a first direction vector corresponding to the corresponding subset based on the adjacent points and the first farther point; a second farther point determination subunit, used to determine a second farther point in a root subset that is relatively farther from the adjacent points based on the adjacent points; a second direction vector construction subunit, used to construct a second direction vector corresponding to the root subset based on the adjacent points and the second farther point; and an angle calculation subunit, used to calculate the angle between the corresponding subset and the root subset based on the first direction vector and the second direction vector.

[0099] According to an implementation manner of an embodiment of the present invention, the leaf subset determination unit is specifically configured to set the corresponding subset as a leaf subset if the angle is greater than 90 degrees, and otherwise not set the corresponding subset as a leaf subset.

[0100] According to an implementation manner of an embodiment of the present invention, the portal vein determination module 705 includes: a maximum point and a minimum point determination submodule, which is used to obtain the maximum point and the minimum point in each subset according to the pixel point position; an extended image data determination submodule, which is used to determine an extended image data corresponding to each subset according to the maximum point and the minimum point; and a portal vein determination submodule, which is used to determine the second image data that has an intersection with the extended image data as the image data corresponding to the portal vein.

[0101] According to a third aspect of an embodiment of the present invention, a computer storage medium includes a group of computer executable instructions, and when the instructions are executed, they are used to execute any one of the above-mentioned image processing methods.

[0102] It should be pointed out here that the above description of the embodiment of the image processing device and the above description of the embodiment of the computer storage medium are similar to the description of the aforementioned method embodiment and have similar beneficial effects as the aforementioned method embodiment, so they are not described in detail. For the technical details that have not been disclosed in the description of the embodiment of the image processing device and the description of the embodiment of the computer storage medium of the present invention, please refer to the description of the aforementioned method embodiment of the present invention for understanding, and in order to save space, they are not described in detail.

[0103] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0104] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another device, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0105] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0106] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0107] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage medium, read-only memory (ROM), disk or optical disk, etc. Various media that can store program codes.

[0108] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods of each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage media, ROM, magnetic disks or optical disks.

[0109] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. An image processing method, applied to a medical auxiliary diagnosis system, comprising: Acquire first image data corresponding to a liver region and second image data corresponding to a blood vessel region, wherein the blood vessels in the blood vessel region include a hepatic vein and a portal vein; Obtaining third image data corresponding to the centerline of the blood vessel according to the second image data and a centerline calculation method, wherein the third image data includes pixel points; Performing connected domain division on the third image data according to a connected domain algorithm and a first rule to obtain at least two subsets; determining, from the at least two subsets, a portal vein centerline subset corresponding to the portal vein centerline according to the first image data and the second rule, wherein the second rule comprises a rule for determining the portal vein centerline subset according to a positional relationship between the portal vein and a liver region and a structural feature of the portal vein; determining image data corresponding to the portal vein from the second image data according to the portal vein centerline subset, so as to distinguish the second image data into image data corresponding to the portal vein and image data corresponding to the hepatic vein; Determining, according to the first image data and the second rule, a portal vein centerline subset corresponding to the portal vein centerline from the at least two subsets comprises: Determine at least one first subset corresponding to the centerline of the blood vessels outside the liver region according to the first image data and the positional relationship between the pixels, and determine the at least one first subset as the subset corresponding to the centerline of the portal vein; Calculating the distance between each first subset and the first image data and determining the first subset with the smallest distance to the first image data as the subset corresponding to the root of the portal vein; According to the subset corresponding to the portal vein root and the rule for determining the portal vein centerline subset according to the structural characteristics of the portal vein, other subsets in the portal vein centerline subset corresponding to the portal vein centerline are determined.

2. The method according to claim 1, wherein the step of acquiring first image data corresponding to the liver region and second image data corresponding to the blood vessel region comprises: Acquiring original image data, wherein the original image data includes image data corresponding to a liver region and image data corresponding to a blood vessel region; The original image data is input into an image processing model to obtain first image data corresponding to the liver region and second image data corresponding to the blood vessel region, wherein the image processing model is an image processing model trained based on a large amount of training image data based on a deep learning method.

3. The method according to claim 1, wherein the first rule comprises: The number of pixels in the subset is greater than a first threshold and less than a second threshold; Accordingly, the third image data is divided into connected domains according to the connected domain algorithm and the first rule to obtain at least two subsets, including: Performing connected domain division on the third image data according to a connected domain algorithm to obtain at least two connected domain subsets; Detect whether the number of pixel points in each connected domain subset is less than a first threshold value. If so, merge the corresponding connected domain subset into any adjacent connected domain subset to obtain a subset that complies with the first rule. If not, further detect whether the number of pixel points in each connected domain subset is greater than a second threshold value. If so, divide the corresponding connected domain subset into at least two subsets that comply with the first rule. If not, retain the connected domain subset.

4. The method according to claim 1, wherein the determining of other subsets of the portal vein centerline subset corresponding to the portal vein centerline according to the subset corresponding to the portal vein root and the rule for determining the portal vein centerline subset according to the structural characteristics of the portal vein comprises: Setting the subset corresponding to the portal vein root as the root subset; Obtain a subset set adjacent to the root subset, detect whether each subset in the subset set is a subset that has been determined to correspond to the centerline of the portal vein, if so, set the corresponding subset as a leaf subset, if not, further perform corresponding processing according to the number of adjacent subsets of the corresponding subset, wherein, if the number of adjacent subsets is greater than 1, further calculate the angle between the corresponding subset and the root subset and determine whether to set the corresponding subset as a leaf subset according to the angle, and if the number of adjacent subsets is equal to 1, set the corresponding subset as a leaf subset; Check whether the number of the leaf subsets is 0. If so, terminate this execution. If not, set the leaf subsets as root subsets in sequence.

5. The method according to claim 4, wherein the calculating the angle between the corresponding subset and the root subset comprises: Determine adjacent points between the corresponding subset and the root subset; Determine, according to the adjacent points, a first distant point in the corresponding subset that is relatively far from the adjacent points; Constructing a first direction vector corresponding to the corresponding subset according to the adjacent points and the first farther point; Determine, according to the adjacent points, a second farther point in the root subset that is relatively farther from the adjacent point; Constructing a second direction vector corresponding to the root subset according to the adjacent points and the second farther point; The angle between the corresponding subset and the root subset is calculated according to the first direction vector and the second direction vector.

6. The method according to claim 4, wherein determining whether to set the corresponding subset as a leaf subset according to the angle comprises: If the angle is greater than 90 degrees, the corresponding subset is set as a leaf subset, otherwise the corresponding subset is not set as a leaf subset.

7. The method according to claim 1, determining image data corresponding to the portal vein from the second image data according to the portal vein centerline subset, comprising: For each of the portal vein centerline subsets, the following steps are performed: Get the maximum and minimum points in each subset according to the pixel position; Determine an extended image data corresponding to each subset according to the maximum point and the minimum point; The second image data intersecting with the extended image data is determined as the image data corresponding to the portal vein.

8. An image processing device, applied to a dialogue system, comprising: An image data acquisition module, used to acquire first image data corresponding to a liver region and second image data corresponding to a blood vessel region, wherein the blood vessels in the blood vessel region include a hepatic vein and a portal vein; a centerline calculation module, configured to obtain third image data corresponding to the centerline of the blood vessel according to the second image data and a centerline calculation method, wherein the third image data includes pixel points; A connected domain division module, used for performing connected domain division on the third image data according to a connected domain algorithm and a first rule to obtain at least two subsets; a portal vein centerline determination module, configured to determine, from the at least two subsets, a portal vein centerline subset corresponding to the portal vein centerline according to the first image data and a second rule, wherein the second rule includes a rule for determining the portal vein centerline subset according to a positional relationship between the portal vein and a liver region and a structural feature of the portal vein; a portal vein determination module, configured to determine image data corresponding to the portal vein from the second image data according to the portal vein centerline subset, so as to distinguish the second image data into image data corresponding to the portal vein and image data corresponding to the hepatic vein; The portal vein centerline determination module is specifically used to determine at least one first subset corresponding to the centerline of the blood vessels outside the liver area according to the first image data and the positional relationship between the pixel points, and determine the at least one first subset as the subset corresponding to the portal vein centerline; calculate the distance between each first subset and the first image data and determine the first subset with the smallest distance to the first image data as the subset corresponding to the portal vein root; determine other subsets in the portal vein centerline subset corresponding to the portal vein centerline according to the subset corresponding to the portal vein root and the rule for determining the portal vein centerline subset according to the structural characteristics of the portal vein.

9. A computer storage medium, comprising a set of computer executable instructions, for executing the method of any one of claims 1 to 7 when the instructions are executed.

Images