Image processing method, image processing device and computer readable storage medium
Through morphological operations and Heisen matrix technology, the problems of void filling and noise interference in medical image segmentation are solved, and more accurate and efficient organ extraction is achieved.
Patent Information
- Application Number
- CN202310618933.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-29
- Publication Date
- 2024-12-03
AI Technical Summary
The existing medical image segmentation technology has problems such as poor segmentation effect, high computational overhead and susceptible to noise, making it difficult to accurately extract specific organ areas.
Through morphological operations, the target void is determined and the expansion coefficient is set according to its size, the expansion operation is performed to fill the void, form the target area, and combined with the Heisen matrix and the region growth algorithm, the impurity parts are extracted and removed to improve the accuracy of the extraction results.
Effectively fill the gaps in medical images, ensure the accuracy and completeness of organ extraction results, reduce noise interference, and improve segmentation efficiency.
Smart Images

Figure CN119067904A_ABST
Abstract
Description
[Technical field]
[0001] The present application relates to the field of image processing technology, and in particular to an image processing method, an image processing device, and a computer-readable storage medium. [Background technology]
[0002] Today's medical imaging technologies, such as computed tomography angiography and magnetic resonance angiography, are widely used. They can provide doctors with medical images of internal organs or other specific parts of the human body, thus greatly facilitating doctors' precise treatment and diagnosis. However, medical images usually contain many different tissues and organs, which can interfere with doctors' diagnosis and treatment and make it difficult to focus on specific target organs.
[0003] Therefore, there are some automatic or semi-automatic image segmentation technology solutions to help segment one or more independent desired organ parts from the complete medical image, or exclude those undesirable factors. However, the existing image segmentation solutions still have many defects, such as poor segmentation effect, high computational cost, and susceptibility to noise. [Summary of the invention]
[0004] The embodiments of the present application provide an image processing method, an image processing device, and a computer-readable storage medium, aiming to solve at least some of the defects of the existing image segmentation technology.
[0005] In a first aspect, the embodiment of the present application provides the following technical solution: an image processing method. The method includes: segmenting to obtain a first organ image in a medical image; performing a morphological operation on the first organ image to obtain a target area; extracting a second organ image in the target area; wherein the performing a morphological operation on the first organ image to obtain the target area specifically includes: determining a target gap in the first organ image according to a preset gap screening standard; determining a corresponding expansion coefficient according to the size of the target gap; and performing an expansion operation on the first organ image based on the expansion coefficient to fill at least a portion of the gap in the first organ image.
[0006] Optionally, determining the target gap in the first organ image according to a preset gap screening criterion specifically includes: searching and determining N inflection points along the boundary of the first organ image; N is a natural number; screening and determining the target inflection point among the N inflection points, wherein the target inflection point is divided into concave points and convex points; and determining the target gap according to the position distribution of the convex points and concave points.
[0007] Optionally, searching and determining N inflection points along the boundary of the first organ image specifically includes: starting from a preset 1st node, determining the 2nd to Kth nodes in sequence along the boundary of the first organ image; among the K nodes, searching for nodes where the boundary direction changes as the inflection points; wherein the boundary direction is determined according to the angle between two adjacent plane vectors; among the two adjacent plane vectors, one plane vector is composed of the k-1th node and the kth node, and the other plane vector is composed of the kth node and the k+1th node.
[0008] Optionally, the screening and determining of the target inflection point among the N inflection points specifically includes: determining a plurality of target inflection points based on the distances between the N inflection points; dividing the target inflection points into convex points or concave points based on the relative direction of each of the target inflection points; wherein the target inflection point is a node on the boundary of the organ unit, the distance between which and two adjacent inflection points meets a preset distance standard.
[0009] Optionally, determining the target gap according to the position distribution of the convex points and the concave points specifically includes: calculating the Euclidean distance between the concave points and the convex points; matching and obtaining multiple candidate gaps according to the Euclidean distances between the concave points and the different convex points; each of the candidate gaps is composed of at least one concave point and at least two convex points; calculating the sum of the Euclidean distances of each of the candidate gaps; the sum of the Euclidean distances is: the superposition of the Euclidean distances between the convex points and the concave points; and determining the target gap from the multiple candidate gaps according to the sum of the Euclidean distances.
[0010] Optionally, determining the corresponding expansion coefficient based on the size of the target gap specifically includes: calculating the gap width and gap depth of the gap; when the gap width is greater than the gap depth, determining the expansion coefficient proportional to the gap width; when the gap depth is greater than the recess width, determining the expansion coefficient proportional to the gap depth.
[0011] Optionally, performing morphological operation on the first organ image to obtain the target area also includes: determining a corresponding erosion coefficient based on the expansion coefficient; and performing an erosion operation on the first organ image after the expansion operation based on the erosion coefficient; wherein the erosion coefficient is greater than the expansion coefficient, and the difference between the erosion coefficient and the expansion coefficient is within a preset difference range.
[0012] Optionally, extracting the second organ image in the target area specifically includes: performing image denoising processing on the target area to obtain a denoised image; and extracting the second organ image from the denoised image based on a Hessian matrix.
[0013] Optionally, the method further includes: performing image post-processing on the second organ image to obtain an output image of the second organ; wherein the image post-processing includes one or more of the following: expanding the second organ image; removing impurity parts of the second organ image that do not belong to the second organ; and removing a third organ image in the second organ image.
[0014] Optionally, the method for performing image post-processing on the second organ image specifically includes: expanding the second organ image; removing impurity parts of the second organ image that do not belong to the second organ in the expanded second organ image; and removing the third organ image in the expanded second organ image.
[0015] Optionally, the expanding the second organ image specifically includes: delimiting a seed point selection area; determining a plurality of boundary pixel points from all pixel points within an overlapping range between the target area and the seed point selection area; the boundary pixel points are: pixel points whose preset neighborhood range exceeds the target area; adding the boundary pixel points to a second organ pixel queue in sequence; sequentially extracting pixel points from the second organ pixel queue, and performing a region growing operation on the extracted pixel points; and obtaining an expanded second organ image when there are no pixels to be extracted from the second organ pixel queue.
[0016] Optionally, the region growing operation includes: when there is a neighborhood pixel point that meets the second organ screening criteria in a preset neighborhood range of the extracted pixel point, determining the neighborhood pixel point as a pixel belonging to the second organ; adding the pixel belonging to the second organ to the second organ pixel queue; and using the pixel belonging to the second organ as a seed point, performing three-dimensional expansion by region growing.
[0017] Optionally, the seed point selection area is an area formed by expanding outward with a preset radius length based on the center point of the target area; the preset neighborhood range is 26 neighborhoods; the preset radius length is proportional to the distance between the center point and the target gap, so that the expanded second organ image falls within the range of the seed point selection area.
[0018] Optionally, removing the impurity part of the second organ image that does not belong to the second organ specifically includes: determining all connected areas in the second organ image and the circumscribed cube corresponding to each of the connected areas; calculating the number of pixels in the connected area and the volume of the corresponding circumscribed cube; determining the ratio between the volume and the number of pixels; and removing the connected area when the ratio is greater than a preset threshold.
[0019] Optionally, the removing of impurity parts in the second organ image that do not belong to the second organ specifically includes: determining connected areas in the second organ image; counting the number of pixels in each of the connected areas respectively; removing the connected areas when the number of pixels is less than a preset threshold; determining a circumscribed cube corresponding to each of the remaining connected areas; calculating the number of pixels in the remaining connected areas and the volume of the corresponding circumscribed cube; determining a ratio between the volume and the number of pixels; and removing the connected areas when the ratio is greater than a preset threshold.
[0020] Optionally, removing the third organ image from the second organ image specifically includes: acquiring the third organ image; performing a dilation operation on the third organ image to obtain an expanded image of the third organ; comparing the expanded image of the third organ with the second organ image to determine an overlapping portion therebetween; and removing the overlapping portion from the second organ image.
[0021] Optionally, the third organ is a trachea; the second organ is a pulmonary blood vessel; and the dilation operation is performed on the image of the third organ to obtain an expanded image of the third organ, specifically comprising: determining three structural elements for performing the dilation operation in a three-dimensional direction; the structural elements are ellipsoidal; and based on the structural elements, dilation processing is performed on pixels that do not belong to the trachea in sequence to obtain an expanded image of the third organ.
[0022] In a second aspect, the embodiments of the present application provide the following technical solutions: an image processing device. The image processing device includes: at least one processor; and a memory connected to the at least one processor; wherein the memory stores computer program instructions, and the computer program instructions are executed by the at least one processor so that the at least one processor can perform the image processing method as described above.
[0023] In a third aspect, the embodiments of the present application provide the following technical solution: a computer-readable storage medium storing a computer program, wherein the computer program implements the above-mentioned image processing method when executed.
[0024] At least one advantageous aspect of the image processing method provided by the embodiment of the present application is that, through morphological operations, the gaps in the first organ image obtained by preliminary segmentation can be appropriately filled, thereby forming a target area that can fully cover the second organ image, thereby effectively ensuring the accuracy of the second organ extraction result.
Brief Description of the Drawings
[0025] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.
[0026] Figure 1 is a schematic diagram of an image processing device according to an embodiment of the present application;
[0027] Figure 2 A flowchart of an image processing method according to an embodiment of the present application;
[0028] Figure 3 A flow chart of a method for performing morphological operations according to an embodiment of the present application;
[0029] Figure 4 This is a lung image of an embodiment of the present application, showing a situation where no morphological closing operation is performed;
[0030] Figure 5 This is a lung image of an embodiment of the present application, showing the situation after being processed by morphological closing operation;
[0031] Figure 6 A flow chart of a method for determining a target gap according to an embodiment of the present application;
[0032] Figure 7 A schematic diagram showing the principle of searching and determining an inflection point in an embodiment of the present application;
[0033] Figure 8 A schematic diagram showing the principle of searching and determining a target inflection point in an embodiment of the present application;
[0034] Fig. 9 This is a schematic diagram of the principle of distinguishing target inflection points as convex points / concave points according to an embodiment of the present application;
[0035] Fig.10 The lung image of the embodiment of the present application shows the inflection point and the target inflection point on the lung boundary;
[0036] Fig.11 This is a lung image of an embodiment of the present application, showing two depressions in the left lung;
[0037] Fig.12 A flowchart of an image processing method according to another embodiment of the present application;
[0038] Fig.13 This is a flow chart of a method for image post-processing according to an embodiment of the present application, showing a situation where a second organ image is expanded based on a region growing algorithm;
[0039] Fig.14AThis is a flow chart of a method for image post-processing according to an embodiment of the present application, showing a situation of screening impurities based on the number of pixels in a connected domain;
[0040] Fig. 14B This is a flow chart of a method for image post-processing according to an embodiment of the present application, showing a case of screening impurities based on a volume and a pixel number ratio;
[0041] Fig.15 This is a flow chart of a method for image post-processing according to an embodiment of the present application, showing a situation where a third organ included in an image of a second organ is removed based on a dilation operation;
[0042] Fig.16 A pulmonary vascular image after image post-processing according to an embodiment of the present application;
[0043] Fig.17 This is a functional block diagram of an image processing system according to an embodiment of the present application. [Specific implementation method]
[0044] In order to facilitate the understanding of the present application, the present application is described in more detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that when an element is described as "fixed to" another element, it can be directly on another element or there can be one or more centered elements therebetween. When an element is described as "connected" to another element, it can be directly connected to another element or there can be one or more centered elements therebetween. The orientation or positional relationship indicated by the terms "upper", "lower", "inner", "outer", "bottom" and the like used in this specification is based on the orientation or positional relationship shown in the accompanying drawings, only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first", "second", "third" and the like are used only for descriptive purposes and cannot be understood as indicating or implying relative importance.
[0045] Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used in this specification and in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The term "and / or" used in this specification includes any and all combinations of one or more of the related listed items.
[0046] In addition, the technical features involved in the different embodiments of the present application described below can be combined with each other as long as they do not conflict with each other.
[0047] In this application, the term "image" refers to data information captured by an imaging system and / or obtained after one or more data processing methods. It can be stored or processed in the form of electronic signals, and can also be printed and presented on real materials or displayed on a display when necessary. The aforementioned imaging system may include, but is not limited to, one or more combinations of magnetic resonance imaging (MRI), computed tomography (CT), positron emission tomography (PET), etc.
[0048] The term "anatomical structure" refers to the specific situation of tissues and organs in different aspects of biological structures, such as spatial organization morphology, positional relationship, structural characteristics and physiological functions. It can be understood that the image of tissues and organs is a reflection of their anatomical structure, and the anatomical structure shows the real situation of tissues and organs.
[0049] "Segmentation" means dividing the contents contained in the image according to specific rules to form one or more different parts. "Extraction" means taking out or separating at least a part of the image data within a selectable or extractable range according to specific rules. It is understandable that extraction and segmentation can be used interchangeably in some cases to express the same or similar meanings.
[0050] Figure 1 The image processing device of the embodiment of the present application can be used to perform one or more steps of the image processing method provided in the embodiment of the present application. Figure 1 As shown, the image processing apparatus 10 may include: a processor 11, a memory 12, a communication bus 13, a communication module 14 and a peripheral device 15. Of course, it may omit one or more of the above devices, or add some other types of devices, but is not limited to the above devices.
[0051] The processor 11 may be any suitable processor or processor group. Specifically, the processor 11 is a multi-core processor capable of multi-threaded processing. The memory 12 may be any suitable storage device, such as ROM, RAM, flash memory or large-capacity mechanical storage device, such as CD-ROM, hard disk, etc. The memory 12 is used to store computer running programs executed by the processor 11 and preset for various data processing.
[0052] The communication module 14 may include some network interaction devices for establishing a communication connection through a connection network to establish a data transmission channel between the electronic device and the outside. The peripheral device 15 includes, but is not limited to: various types of sensors and input / output devices (e.g., keyboard, mouse and display). The display may include any type of computer playback device or electronic playback device (such as a CRT or LCD-based device) for displaying information to the user.
[0053] In actual application, the processor 11 and the memory 12 communicate with each other through the communication bus 13. The processor 11 can call the computer running program in the memory 12, perform some image processing operations, process images from different resources to determine the parts of interest therein, and further provide them to the display for back-end operations. For example, the processor 11 can implement some or all steps of the image processing method described or disclosed in the embodiment of the present application on the received CT image, form pulmonary vascular image data and present it on the display.
[0054] According to the inventive concept provided by the embodiment of the present application, it can be generally applied to the image segmentation processing of medical images. However, for the convenience of description and understanding, the following will mainly use the pulmonary vascular segmentation in CT images as an example to describe the present application.
[0055] Figure 2 This is a flowchart of the image processing method provided by the embodiment of the present application. Figure 2 As shown, the image processing method may include the following steps:
[0056] S100: Segment and obtain a first organ image in a medical image.
[0057] Among them, the medical image refers to image information containing at least a part of human organ tissue. The first organ can be a specific human tissue organ. For example, the lung or the liver. It should be noted that the first organ can also be a target or region of interest determined according to actual needs. For example, in other scenarios, the first organ can also refer to the left lung located on the left side of the chest cavity or the right lung located on the right side of the chest cavity.
[0058] In the present application, any suitable type of algorithm can be selected to perform preliminary segmentation on the medical image according to the actual needs to obtain the required first organ image. Alternatively, a deep learning network (e.g., a U-Net network) can be used to complete the aforementioned segmentation operation.
[0059] S200: Perform morphological operation on the first organ image to obtain a target area.
[0060] Among them, morphological operation refers to the operation of using various structural elements to operate on the image to strengthen or weaken the morphological features of the local area of the image, so as to obtain some images that have been processed by specific morphological transformation. "Target area" refers to the result obtained after the first organ image is subjected to morphological operation. Because it includes the area range extracted from the second organ image, it is called the target area. In other embodiments, the target area may also be called a "region of interest" or a "mask" based on its characteristics.
[0061] In the process of implementing the present application, the applicant has found that: in the first organ image obtained by segmentation in step S100, there are usually many gaps. The second organ, which is the real extraction object, is likely to be distributed and exist in these gaps in terms of anatomical structure. Therefore, it is necessary to fully cover the area where the second organ exists as much as possible to avoid errors and omissions in the final extraction result, and it is necessary to fill these gaps to form a complete area.
[0062] For example, in the segmentation process of lung CT images, segmentation and extraction are usually performed based on HU values, that is, through edge detection or similar methods, the part with a smaller HU value in the CT image is extracted from the image to obtain the corresponding lung image.
[0063] However, the HU value of the root of the pulmonary blood vessels is significantly greater than the aforementioned segmented and extracted part. This means that during the segmentation and extraction process, the image corresponding to the root of the pulmonary blood vessels will not be segmented into the lung image, resulting in gaps in the segmented lung image.
[0064] Therefore, if these gaps are not processed, the roots of the pulmonary vessels will be missing when the second organ image is subsequently extracted, affecting the accuracy of the pulmonary vessel extraction results.
[0065] S300: extracting a second organ image in the target area.
[0066] Among them, the second organ is a tissue organ that is different from the first organ but is mainly distributed in the first organ. For example, when the first organ is the lung, the second organ may be a pulmonary blood vessel; or when the first organ is the liver, the second organ may be a liver blood vessel. However, it should be noted that the aforementioned first organ and the second organ are not limited to anatomical structures in a strict sense. It can be determined according to actual clinical treatment needs, as long as it can be distinguished. For example, the hepatic blood vessels may be a collection of the hepatic outflow blood vessels and the hepatic inflow blood vessels.
[0067] It can be seen from the above comparison that the image processing method provided in the embodiment of the present application can effectively ensure the accuracy of the blood vessel segmentation and extraction results.
[0068] Furthermore, if Figure 3 As shown, the morphological operation may specifically include the following operation steps:
[0069] S210: Determine a target gap in the first organ image according to a preset gap screening standard.
[0070] The gap screening criteria may be a screening criteria set based on the size of the gap. The specific standard description and definition method may be set according to the actual needs, and are not specifically limited here. For example, the target gap may be the largest gap in the organ image.
[0071] In this application, the term "gap" is used to represent a discontinuous area in the first organ image. Typically, the gap usually includes two different types of areas, "depression" and "hole". Among them, "depression" refers to an area that is sunken inward from the edge of the outer contour of the image, while a hole refers to an area located in the image, completely surrounded by the image, and has no internal connection.
[0072] S220, determining a corresponding expansion coefficient according to the size of the target gap.
[0073] The expansion coefficient refers to the size of the structural element when the expansion operation is performed. A larger expansion coefficient indicates a larger expansion effect, and vice versa.
[0074] S230 . Perform a dilation operation on the first organ image based on the dilation coefficient to fill at least a portion of the gaps in the organ unit.
[0075] As mentioned above, since the expansion coefficient is determined based on the target gap, it is possible to ensure that the depressions or holes in the first organ image are successfully filled by selecting a suitable target gap.
[0076] For example, since the depression formed by the root of the pulmonary blood vessels is necessarily larger than the depression or hole extending into the lungs, when the depression with the largest size formed by the root of the pulmonary blood vessels is selected as the target gap, the expansion operation can ensure that the depression or hole in the lung image can be filled to obtain the required target area.
[0077] Specifically, the aforementioned expansion coefficient can be calculated and determined in the following manner:
[0078] First, calculate the gap width and gap depth of the target gap. The gap width and gap depth refer to the size of the target gap in two different directions. For example,
[0079] Assuming that the target gap is a depression consisting of a concave point and two convex points, the gap width refers to the Euclidean distance between the two convex points, and the gap depth is the Euclidean distance between the concave point and the vector formed by the two convex points. Then, the values of the gap width and the gap depth are compared to determine the larger value. Finally, the proportional expansion coefficient is determined based on the comparison result.
[0080] When the gap width is greater than the gap depth, an expansion coefficient proportional to the gap width is determined. When the gap depth is greater than the recess width, an expansion coefficient proportional to the gap depth is determined. "Proportional" means that there is a predetermined proportional relationship between two values. The proportional relationship can be set according to the actual needs, such as two-thirds.
[0081] In other application scenarios, such as in the segmentation and extraction of liver blood vessels, the target gaps are usually determined as holes. The aforementioned gap width and gap depth can also use two different parameters that can represent the hole size. These parameters that can represent the hole size include but are not limited to: the equivalent diameter of the hole, the diameter of the minimum circumscribed circle, or the length or width of the minimum circumscribed rectangle, etc.
[0082] In some other embodiments, in addition to the dilation operation, the aforementioned morphological operation may further perform an additional image processing step of an erosion operation after the dilation operation is completed. Typically, such a method of first performing the dilation operation and then performing the erosion operation may also be referred to as "morphological closing processing".
[0083] The erosion operation is an operation that shrinks or ablates the target object to remove or reduce its edges or details. It is performed based on a set erosion coefficient and can smooth the edges of the target area, thereby helping to reduce errors in subsequent image processing.
[0084] For example, in the application scenario of pulmonary vascular segmentation, the segmentation and extraction results will produce edge-vessel-like strip impurities (hereinafter referred to as edge impurities). These strip impurities have the characteristics of being similar to blood vessels and connected to pulmonary vessels, making it difficult to remove them using appropriate image processing methods. The aforementioned corrosion operation can effectively remove such edge impurities by reasonably shrinking the target area, thereby improving the accuracy of segmentation and extraction.
[0085] Specifically, the corrosion coefficient of the aforementioned corrosion operation can be determined based on the already determined expansion coefficient. For example, the corrosion coefficient can be set to be slightly larger than the expansion coefficient by 3-5 pixels. Of course, the preset difference between the corrosion coefficient and the expansion coefficient can also be adjusted or set according to the needs of the actual situation, and is not limited to the aforementioned 3-5 pixels.
[0086] Figure 4 The lung image obtained by segmentation in the embodiment of the present application and not processed by morphological closing operation. Figure 5 for Figure 4 Schematic diagram of the lung target area obtained after morphological closing operation. Figure 4 and 5As shown in the figure, in the target lung region obtained after the morphological closing operation, the main holes and depressions are properly filled and the edges are properly adjusted, which can provide a more accurate position range to ensure the accuracy of subsequent pulmonary vascular extraction results.
[0087] In other scenarios, the aforementioned image processing method can also be applied to the segmentation and extraction of hepatic blood vessels. In other words, the first organ is the liver, and the second organ is the hepatic blood vessels. First, the liver image is segmented in the medical image. Then, a morphological operation is performed on the liver image to obtain the corresponding target area, and the target area is applied to the subsequent hepatic blood vessel extraction.
[0088] As mentioned above, appropriate screening criteria are needed to help determine the target gaps required to ensure that the expansion operation can successfully complete the filling of the gaps. Figure 6 , the specific selection process of the target gap is explained.
[0089] Figure 6 A flow chart of a method for determining a target gap provided in an embodiment of the present application. Figure 6 As shown, the target space in the organ unit can be selected and determined by the following method steps:
[0090] S211. Search and determine N inflection points along the boundary of the organ unit.
[0091] Wherein, N is a natural number, which is used to represent the total number of inflection points. Its specific value is searched and determined according to the actual situation. "Boundary" refers to the edge of the first organ in the medical image, which is used to divide the boundary between the first organ and other contents in the image. In other words, with the boundary as the boundary, the part on one side belongs to the first organ and the part on the other side does not belong to the first organ.
[0092] An "inflection point" is a point at which the tangent direction of a curve changes suddenly or the direction of the curve changes. In other words, there is a change between the boundary direction before the inflection point and the boundary direction after the inflection point.
[0093] There may be various ways to search and determine the inflection point. In order to fully illustrate the inventive idea of the present application, a specific inflection point search method is described in detail. However, those skilled in the art will appreciate that other different inflection point search methods may be used or the following inflection point search methods may be adjusted or replaced based on actual needs, and are not limited to the search methods described in the present application.
[0094] First, starting from the preset first node, the second to the Kth nodes can be determined in sequence along the aforementioned boundary. Among them, "node" refers to the pixel point located on the boundary; K is an arbitrary positive integer, which is used to represent the total number of nodes. The specific method of selecting nodes or the density of node selection can be arbitrarily selected according to the needs of the actual situation. For example, each pixel point on the boundary can be selected as a node in units of pixels. Alternatively, a node can be selected after every certain number of pixel points. In addition, the sequential numbering of nodes is for the convenience of description and distinction, and is not used to make any other limitations on the nodes. The first node as the starting position of the search can be arbitrarily selected or set according to the needs of the actual situation.
[0095] Then, among the K nodes, search for nodes where the boundary direction changes as inflection points. The "boundary direction" can be determined based on the angle between two adjacent plane vectors. Each plane vector is composed of the adjacent kth node and k+1th node; k is a positive integer between 1 and K. For example, Figure 7 As shown, among the four adjacent nodes from P1 to P4, three plane vectors P1P2, P2P3 and P3P4 are formed.
[0096] Based on the definition of "cross product", those skilled in the art can understand that by calculating the cross product between two vectors, the angle between the two vectors can be determined. When the cross product is greater than zero, the angle between the two vectors is positive, and when the cross product is less than zero, the angle between the two vectors is negative. Therefore, for multiple consecutive plane vectors, the change in sign after the cross product between two successive vectors actually indicates the direction change or trend of the boundary.
[0097] Specifically, for a closed boundary, K nodes will form K plane vectors, and the adjacent plane vectors will calculate their cross products to obtain the corresponding K values. When the value remains positive or negative, it means that the direction of the boundary has not changed. When the value changes from positive to negative, or from negative to positive, it means that the direction of the boundary has changed.
[0098] In actual operation, in order to conveniently determine the change of signs, we can perform "XOR operation" on two adjacent cross product signs so that their change can be represented by binary numbers. For example, when the signs are the same, the result of the XOR operation is 0; and when the signs are opposite, the result of the XOR operation is 1.
[0099] Accordingly, starting from the first node, we check the XOR result. When the following characteristics are met, we can determine that the node is a turning point: a number of consecutive 0s appear (which indicates that the trend of the boundary has not changed), followed by a value 1, and finally multiple values 0 appear consecutively after the value 1.
[0100] S212, screening and determining the concave points and convex points among the N inflection points.
[0101] Among them, concave points and convex points are special nodes that appear in the boundary. In this article, for the sake of simplicity, these special nodes (i.e., concave points and convex points) are collectively referred to as target inflection points. Specifically, a concave point refers to a shape that bulges outward at a point in the boundary curve, and this point is called a concave point. Correspondingly, a convex point refers to a shape that is concave inward at a point in the boundary curve, and this point is called a convex point.
[0102] Specifically, firstly, according to the distances between the N inflection points, nodes meeting a preset distance standard may be determined as target inflection points, and then according to the directionality of each target inflection point, the target inflection point may be divided into a convex point or a concave point.
[0103] The preset distance standard can be set according to the actual needs. Figure 8 As shown, the point A with the largest distance between two adjacent inflection points M and N can be determined as a target inflection point. Of course, other appropriate distance standards can also be set to screen and determine the target inflection point.
[0104] In addition, there are various ways to determine whether the target inflection point is a convex point or a concave point. Fig. 9 , describes in detail a specific concave / convex point judgment method. However, those skilled in the art will appreciate that other different judgment methods may be adopted or the following judgment methods may be adjusted or replaced based on actual needs, and are not limited to those described in this application.
[0105] For the three nodes P1, P2 and P3, there is a vector principle expressed by the following formula (5):
[0106]
[0107] Among them, when P1, P2 and P3 are clockwise, S is a negative value; when P1, P2 and P3 are counterclockwise, S is a positive value.
[0108] Combining the characteristics of the above formula (5), we can determine: Fig. 9 As shown, when the node P3 is located on the left side of the vector P1P2, S is a positive value, and when the node P3 is located on the right side of the vector P1P2, S is a negative value. When the node P3 is located on the vector P1P2, S is 0.
[0109] Thus, the directionality of the target inflection point can be determined by the value of S, and whether it should be a convex point or a concave point. In this embodiment, the term "directivity" is used to represent the position information of the target inflection point relative to the vector.
[0110] S213, determining the target gap according to the position distribution of the convex points and the concave points.
[0111] The position distribution represents the relative position relationship between each concave point and convex point, which can be represented by any suitable type of data information. "Position distribution" is used here to illustrate and indicate that the position between each concave point and convex point is known or can be obtained by calculation.
[0112] As mentioned above, an organ unit may have multiple gaps (such as depressions or holes). These depression and hole structures should be composed of multiple concave points and convex points. In other words, the location where the depression or hole is formed will generate and form the aforementioned concave points and convex points at the boundary. Therefore, in combination with the actual application situation, appropriate screening and judgment conditions can be set to select and determine the concave points and convex points that constitute the target gap to determine the specific target gap.
[0113] In order to fully illustrate the inventive concept of the present application, the following takes the left lung as the first organ image, and the target gap to be determined is the depression formed by the root of the pulmonary blood vessels as an example, and describes in detail the specific method of determining the target gap. However, those skilled in the art can understand that the specific screening and judgment criteria can also be adjusted or replaced based on actual needs, and are not limited to the description of the present application.
[0114] For example, Fig.10 As shown, multiple concave points, convex points and other inflection points can be searched and determined on the boundary of the left lung. Based on the existing convex points and inflection points, the Euclidean distance between the concave points and all convex points can be calculated.
[0115] Then, multiple candidate concavities are obtained by matching according to the Euclidean distance between the concave point and different convex points. Fig.10 , the target gap can be roughly considered to be composed of one concave point and two convex points. Therefore, the concave point can be used as a basis to find two convex points that match it as a depression. The matching convex points can be considered to be the two convex points that are closest to the Euclidean distance between the concave points. In other words, each depression can be composed of a concave point and a pair of two convex points that are closest to the Euclidean distance between the concave points.
[0116] Alternatively, other numbers of concave points and convex points may be matched to form the required depressions based on geometry or the like according to actual application scenarios, and the matching method to form the target gap may also be adjusted accordingly.
[0117] In some scenarios, the number of depressions formed by pairing may be only one, so it can be directly determined as the target gap. However, in more cases, the number of depressions is not limited to one. For example, Fig.11As shown, other smaller depressions C may appear on the border of the left lung, but they are not the ideal target spaces A.
[0118] Therefore, the screening criteria of the sum of Euclidean distances can be added to further screen and determine the target gaps. For example, the sum of the Euclidean distances of all the depressions can be compared, and the depression with the largest sum of the Euclidean distances can be used as the target gap.
[0119] Among them, the "sum of Euclidean distances" refers to the result obtained by superimposing the Euclidean distances between the convex points and the concave points. From a geometric point of view, the maximum sum of Euclidean distances represents the maximum depression depth that the concave point can achieve when the convex point is determined. Therefore, by selecting the candidate concave with the largest sum of Euclidean distances, it can help exclude smaller concave C, so as to obtain and determine the true target gap A.
[0120] Alternatively, in addition to the aforementioned screening and exclusion criterion of “the sum of the Euclidean distances”, a screening criterion of “the Euclidean distance between two salient points is less than or equal to a preset threshold” may be added.
[0121] The preset threshold is set according to the actual application. For example, in combination with the anatomical structure, the width of the root depression of the pulmonary blood vessels is usually limited and has a certain proportional relationship with the size of the lungs. Therefore, according to the proportional relationship, a suitable threshold can be set to screen those candidate depressions that are obviously not the target gaps.
[0122] In actual application, the maximum circumscribed rectangular box of the left lung on the two-dimensional plane can be first determined. Then, the maximum length of the maximum circumscribed rectangular box is determined, and one third of the maximum length is used as the aforementioned preset threshold. When the Euclidean distance between two convex points of the candidate depression is greater than or equal to the preset threshold, it is excluded.
[0123] It should be noted that the aforementioned two screening criteria of "sum of Euclidean distances" and "preset threshold" can be used simultaneously or independently, or can be used in combination with other appropriate screening criteria without conflict to meet different practical application needs.
[0124] In other embodiments, when the liver is used as the first organ image, the target gap to be determined may be a hole formed by the vena cava located inside the liver. Therefore, in the process of searching and determining the target gap, the adjustment is made as follows: the hole contained in the liver image is determined based on the convex points and concave points, and then the target gap is determined by a similar method such as the Euclidean distance. The specific adjustment method is well known to those skilled in the art and will not be described in detail here.
[0125] Specifically, the aforementioned extraction operation may include the following steps:
[0126] First, the target area is subjected to image denoising to obtain a denoised image.
[0127] In the imaging process, the digital image will generate noise to varying degrees due to the influence and interference of factors such as the imaging device and the transmission method. The noise influence in the image can be eliminated as much as possible through a suitable noise reduction filter processing method. In this embodiment, the median filter method is used to perform image noise reduction processing to obtain a noise-reduced image after eliminating noise interference. Alternatively, other image noise reduction methods, such as Gaussian filtering or mean filtering, can be selected according to the actual needs.
[0128] Secondly, based on the Hessian matrix, the second organ image is extracted from the denoised image.
[0129] Among them, the Hessian Matrix is a matrix composed of second-order partial derivatives, which can help evaluate and describe the quadratic changes in pixel values in an image. Therefore, by calculating the eigenvectors and eigenvalues of the Hessian Matrix, the structural features of the target object to be extracted (the changes in the grayscale of the image pixels) can be reflected and described, thereby automatically extracting the target object (for example, texture information such as blood vessels) from the denoised image.
[0130] In order to fully describe the present application, the automatic extraction and segmentation of the second organ image is described in detail below by taking the extraction of pulmonary blood vessels as an example.
[0131] First, the local features of the sample image at a certain pixel x0 can be described by the Taylor expansion (Taylor) shown in the following formula (1):
[0132]
[0133] Among them, δ x0 Indicates the increment, Represents the gradient of the image at pixel x0. 0,s Represents the Hessian matrix of the image at pixel x0.
[0134] In three-dimensional space, the Hessian matrix at pixel x0 is as shown in the following formula (2):
[0135]
[0136] The relationship between the eigenvectors and eigenvalues of the aforementioned Hessian matrix can be expressed by the following formula (3):
[0137]
[0138] In the three-dimensional space, each eigenvalue of the Hessian matrix corresponds to an eigenvector, and the eigenvectors are mutually orthogonal in the three-dimensional space. λ1, λ2, and λ3 represent the directions of the eigenvectors corresponding to the three eigenvalues of the Hessian matrix, λ1 is parallel to the normal direction of the blood vessel, λ2, λ3 are perpendicular to the normal direction of the blood vessel, and λ1 and λ2 and λ3 are orthogonal to each other and perpendicular to each other.
[0139] Then, according to the anatomical structure characteristics of the pulmonary blood vessels, the eigenvalue corresponding to its local tubular structure is: λ1 is very small and close to 0, while the absolute values of λ2 and λ3 are both large and approximately equal. This eigenvalue can be expressed by the following formula (4):
[0140] |λ1|≈0<<|λ2|≈|λ3| (4)
[0141] Finally, based on the eigenvalues determined above, the Hessian matrix can be used to automatically detect the denoised image, determine and automatically extract the pixels that meet the eigenvalues as pulmonary vascular pixels, and thus form the pulmonary vascular three-dimensional image to be extracted.
[0142] Fig.12 Another embodiment of the present application provides an image processing method. Fig.12 As shown, compared with the above-mentioned embodiment, in addition to the second organ image obtained in the above-mentioned steps S100 to S300, the image processing method further includes the following steps:
[0143] S400: performing image post-processing on the second organ image to obtain an output image of the second organ.
[0144] Among them, "image post-processing" refers to a variety of processing operations used to further improve the accuracy of the extraction results and eliminate impurity data caused by interference factors such as contrast agent concentration, partial volume effect, noise, etc. For example, in addition to normal pulmonary vascular data, other non-vascular data are included in the pulmonary vascular image.
[0145] The aforementioned image post-processing may include one or more of the following: expanding the second organ image; removing impurity parts of the second organ image that do not belong to the second organ; and removing the third organ image in the second organ image.
[0146] In order to fully describe the aforementioned image post-processing process, the image post-processing of pulmonary vascular images is taken as an example to specifically describe the aforementioned multiple image post-processing methods.
[0147] 1) About expanding the second organ image:
[0148] In some specific application scenarios, since the selectable range of the extracted second organ image is limited to the target area, some partial tissue structures that belong to the second organ in anatomical structure may be located outside the target area and need to be supplemented and improved through appropriate expansion methods.
[0149] For example, blood vessels rapidly branch and spread after entering the lungs from the heart through the hilum. The pulmonary vascular image data extracted within the aforementioned target area may have a phenomenon called "broken branches" (i.e., the pulmonary vascular image data does not include or includes a portion of the root of the pulmonary blood vessels). Therefore, based on the pulmonary vascular image obtained in step S300, its root can be expanded so that the branches of the same vascular tree merge into a whole tree, eliminating the broken branch phenomenon.
[0150] In the process of implementing the present application, the applicant discovered that when the sampled image provided is a CT image, the root of the pulmonary blood vessel can be expanded in the existing target region using a regional growing algorithm based on the characteristic that the CT value of the root region of the pulmonary blood vessel is larger while the CT values of other regions are smaller. Alternatively, based on the inventive concept disclosed in the present application, a growing algorithm based on breadth-first search can also be used to complete the expansion of the root of the pulmonary blood vessel.
[0151] In order to fully describe the expansion process of this application, the following is a detailed description based on the region growing algorithm as an example. Fig.13 As shown, it can generally include the following steps:
[0152] S411, defining a seed point selection area. The seed point selection area refers to an area where the second organ may appear, and by setting a reasonable area, it is possible to avoid expanding a part that does not belong to the second organ.
[0153] For example, the seed point selection area can be an area formed by expanding outward with a preset radius length based on the center point of the target area. For example, the coordinates of all voxels in the lung area can be calculated first, and then the average value of the coordinates can be calculated as the aforementioned center point. Then, the Euclidean distance between the center point and the two convex points in the aforementioned depression is calculated, and the size of 1.5 times the Euclidean distance is taken as the aforementioned preset radius length.
[0154] S412, determining a plurality of boundary pixel points from all pixel points within the overlapping range of the target area and the seed point selection area.
[0155] The boundary pixel point is a pixel point whose preset neighborhood range exceeds the target area and is located at the boundary of the target area. It can be obtained by searching in a traversal manner. Optionally, the preset neighborhood range can be a 26-neighborhood, that is, including 26 adjacent pixels adjacent to the target pixel.
[0156] In this way, pixel points that belong to the pulmonary blood vessels and are located at the boundary of the target area can be found as seed points (ie, boundary pixel points).
[0157] S413: Add the boundary pixels to the second organ pixel queue in sequence.
[0158] The second organ pixel queue can be considered as a set with an arrangement order. It is arranged in pixel units in the order of addition. Each time a new seed point is added, it is added from the end of the second organ pixel queue, and each time a region is grown, it is taken from the head of the second organ pixel queue to achieve orderly growth.
[0159] S414: Take out pixel points in the second organ pixel queue in order, and perform a region growing operation on the taken out pixel points.
[0160] Here, "sequentially" means that the operation of taking out the pixel from the head of the second organ queue each time, that is, the seed point that enters the queue first will be first subjected to region growing. The region growing can be based on the 26-neighborhood growth of the aforementioned target region.
[0161] Specifically, the process of growing the 26 neighborhood regions includes:
[0162] First, it is determined whether there is a neighboring pixel point that meets the second organ screening criteria in the 26 neighborhoods of the extracted pixel point. If so, the neighboring pixel point is determined to be a pixel belonging to the second organ. Then, the neighboring pixel point is added to the second organ pixel queue, and is used as a seed point to perform three-dimensional expansion by region growing.
[0163] Among them, the second organ screening standard is set according to the actual situation. For example, in combination with the characteristics of the pulmonary blood vessels, the second organ screening standard can be set to a specific CT value. When the neighborhood pixel meets the specific CT value, it is determined that it belongs to the pulmonary blood vessel pixel. Of course, in order to avoid excessive expansion, a specific upper limit of the number of pixels can also be set. The upper limit of the number of pixels is an empirical value, which can be set according to the needs of the actual situation to limit the range of the degree of expansion.
[0164] S415: When there are no pixels that can be taken out in the second organ pixel queue, output the expanded second organ image.
[0165] Among them, "no pixel can be taken out" means that the second organ pixel queue is an empty set at this time, and no similar adjacent pixels can be added. Therefore, it can be confirmed that the expansion has been completed and the expanded second organ image has been obtained.
[0166] Furthermore, the applicant discovered in the process of implementing the present application that if the expansion operation is directly performed on the complete first organ image, it is likely to cause the two organ units that were originally independent of each other in the image to adhere to each other, forming a continuous target area. For example, near the top of the lung, the positions of the left lung and the right lung are relatively close. After the morphological closing operation, the left lung and the right lung will be connected to the same target area. However, since the subsequent expansion operation is performed based on the edge of the target area, therefore, incorrect coverage of the target area will result in an erroneous root expansion result.
[0167] In order to solve this problem, the applicant discovered that the adhesion phenomenon can be avoided by performing morphological closing operations on relatively independent organ units in the first organ image respectively, thereby ensuring the accuracy of the target area obtained after the morphological closing operation.
[0168] 2) Regarding removing the impurity part of the second organ image that does not belong to the second organ:
[0169] Affected by many factors such as noise interference, partial volume effect, sampling artifacts, etc., the extracted second organ image may contain not only normal second organs but also some other impurities. Therefore, some appropriate screening criteria can be set to help screen and remove parts that obviously do not belong to the second organ.
[0170] On the one hand, the numerical value of the number of pixels can be used as a screening criterion. Fig.14A The specific calculation method is as follows:
[0171] S421: Determine a connected region in the second organ image.
[0172] Here, "determining" means being able to distinguish and identify each different connected area (in other embodiments, it may also be referred to as a "connected domain"). For example, each connected area may be marked, assigned a number, etc. to distinguish different connected areas.
[0173] S423: Count the number of pixels in each connected region respectively, wherein the number of pixels refers to the total number of pixels contained in the connected region.
[0174] S425: When the number of pixels is less than a preset threshold, remove the connected area.
[0175] The preset threshold is an empirical value, which can be set according to actual needs. For example, pulmonary blood vessels usually have a specific size in terms of anatomical structure.
[0176] Therefore, when the number of pixels is significantly lower than a specific number, it is considered with a high degree of certainty that the image does not belong to the pulmonary blood vessels and is removed from the second organ image.
[0177] On the other hand, the applicant has found that in three-dimensional space, the vascular tree of the pulmonary vessels is usually distributed in a relatively dispersed area, while impurities are usually distributed in a relatively concentrated area and show a clustering phenomenon. Therefore, it is also possible to determine whether the connected area belongs to the pulmonary vessels by the density of pixels.
[0178] The density of the aforementioned pixels can be measured by the pixel volume ratio of the connected area. Fig. 14B As shown, the specific calculation method is as follows:
[0179] S422: Determine all connected regions in the second organ image and the circumscribed cube corresponding to each connected region.
[0180] The circumscribed cube refers to a cube that can just enclose the connected region. Similar to step S421, each connected region can be marked independently for distinction. For example, the connected region can be given a number or a position mark.
[0181] S424, calculating the number of pixels in the connected region and the volume of the corresponding circumscribed cube, and determining the ratio between the volume and the number of pixels of each connected region.
[0182] S426: When the ratio is greater than a preset threshold, remove the connected area.
[0183] Among them, the preset threshold is an empirical value, which can also be set according to the needs of the actual situation. Through the preset threshold, the density of pixels can be measured and judged. When the number of pixels and the volume ratio are greater than the preset threshold, it indicates that the pixels in the current connected area are relatively dense and are likely to be impurities, and they need to be removed from the second organ image. When the number of pixels and the volume ratio are less than the preset threshold, it indicates that the pixels in the current connected area are relatively scattered, and there is a greater degree of confidence that they belong to pulmonary blood vessels and should be retained.
[0184] Alternatively, the two aforementioned screening criteria can also be applied to screen the connected areas at the same time, so as to screen out impurities as much as possible. As mentioned above, the two screening criteria have partially overlapping steps when screening. Therefore, you can first perform Fig.14A The screening criteria shown in the figure are then executed. Fig. 14B The screening criteria shown are used to avoid unnecessary additional computing power consumption.
[0185] 3) Remove the third organ image from the second organ image:
[0186] In medical images, some known tissues and organs often appear together with the second organ to be extracted. These tissues and organs are similar in nature and structure to the second organ and are usually difficult to eliminate through image processing and other methods. In this embodiment, they are called "third organs".
[0187] For example, in the lung CT image data, the trachea and the pulmonary blood vessels are similar and are abundantly distributed in the left lung and the right lung. Since the tracheal wall and the pulmonary blood vessels have similar CT values, the pulmonary blood vessel image data extracted and obtained in step S300 based on edge detection is likely to contain the tracheal wall that is segmented incorrectly.
[0188] Therefore, based on the known anatomical characteristics of the third organ (for example, the thickness of the tracheal wall is usually between 3-5 pixels), the third organ can be specifically excluded from the second organ image by designing targeted image post-processing operations. Fig.15 As shown, the above-mentioned image post-processing operation of removing the third organ may include the following steps:
[0189] S431, acquiring a third organ image.
[0190] In this embodiment, the third organ is used to represent the tissue structure of the trachea which is different from the first organ and the second organ. The third organ image can be obtained by segmenting the sample image in any suitable manner, which is not specifically limited here.
[0191] S432: Perform a dilation operation on the third organ image to obtain an expanded image of the third organ.
[0192] The execution of the expansion operation can help the aforementioned tracheal image to be appropriately expanded to include the tracheal wall. In other words, the expanded image of the third organ is image data including the tracheal wall.
[0193] In some embodiments, the expansion operation may be performed in a three-dimensional direction, and a dilated image of the third organ is obtained by sequentially performing expansion processing on pixels that do not belong to the trachea through three structural elements.
[0194] The expansion process for each pixel may specifically include the following steps: first, the center of the sphere of the structure element is placed at the position of the pixel to be processed; then, when there is a pixel belonging to the trachea within the range covered by the structure element, the pixel to be processed is determined to be a tracheal expansion pixel as part of the expansion image of the third organ. Alternatively, the aforementioned structure element may use an ellipsoidal structure element so that it can be close to the anatomical structure of the trachea. The size of the structure element is determined by the known characteristics of the organ (for example, the thickness of the tracheal wall is usually between 3-5 pixels).
[0195] S433: Compare the expanded image of the third organ and the second organ image, determine an overlapping portion therebetween, and remove the overlapping portion from the second organ image.
[0196] The third organ expansion image obtained after the appropriate expansion operation includes the tracheal wall portion to be removed. Therefore, the overlapping portion between the third organ expansion image and the second organ image can be roughly considered as the tracheal wall. Thus, after marking and deleting the overlapping portion, the effect of removing the tracheal wall in the pulmonary vascular image can be achieved.
[0197] In some embodiments, the execution order of the aforementioned multiple image post-processing operations can be adjusted or changed without determining their specific order. Alternatively, the aforementioned image post-processing operation of "expanding the second organ image" can take precedence over the image post-processing operation of "removing the impurity part of the second organ image that does not belong to the second organ" to avoid removing the part that belongs to the blood vessel but has not yet connected to the main blood vessel to form a blood vessel tree when removing the impurity part. Fig.16 This is a pulmonary vascular image that has undergone image post-processing provided in an embodiment of the present application.
[0198] like Fig.16 As shown, after the aforementioned multiple image post-processing operations, a good segmentation result can be obtained without the influence of impurities, with obvious amplification of the blood vessel roots, so that the left and right pulmonary vessels each merge into the same pulmonary vascular tree.
[0199] Fig.17 The image processing system provided in the embodiment of the present application can be implemented by the aforementioned electronic device through software / hardware or software and hardware in cooperation with each other. Fig.16 As shown, the image processing device 20 includes: an image segmentation module 21 , a target area calculation module 22 and an extraction module 23 .
[0200] The image segmentation module 21 is used to segment and obtain a first organ image in a medical image. The target region calculation module 22 is used to perform morphological operations on the first organ image to obtain a target region. The extraction module 23 is used to extract a second organ image from the target region.
[0201] When performing the morphological closing operation, the target region calculation module 22 is specifically used to: determine a target gap in the first organ image; determine a corresponding expansion coefficient according to the size of the target gap and perform an expansion operation based on the expansion coefficient to fill at least a portion of the gap.
[0202] It should be noted that, in the embodiments of the present application, the functional modules named by functions are taken as examples to describe the method steps to be implemented by the image processing device. Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described device and module can refer to the corresponding process in the aforementioned one or more method embodiments, which will not be repeated here. Those skilled in the art can appreciate that, although in the embodiments of the present application, the functional modules of the image processing device are divided according to the method steps to be performed. However, according to the needs of the actual situation, one or more functional modules in the image processing device of the embodiment of the present application can be split into more functional modules or integrated into fewer functional modules to perform the corresponding method steps.
[0203] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium may be a non-volatile storage medium. The computer-readable storage medium stores a computer program. When the computer program is executed by the processor, one or more method steps in the above-mentioned display control method can be implemented. The complete computer program product is embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing the computer program disclosed in the embodiment of the present application.
[0204] The present application also provides a surgical robot. The surgical robot may include a main operating console, a slave operating device, and the aforementioned image processing device. The image processing device is coupled to the main operating console and the slave operating console, can receive image information from the imaging system, and execute one or more steps of the image processing method in the aforementioned method embodiment.
[0205] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Under the concept of the present application, the technical features in the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other changes in different aspects of the present application as described above, which are not provided in detail for the sake of simplicity. Although the present application has been described in detail with reference to the aforementioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features can be replaced by equivalents. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.
Claims
1. An image processing method, It is characterized in that include: Segmenting and obtaining a first organ image in the medical image; Performing morphological operation on the first organ image to obtain a target area; extracting a second organ image in the target area; The step of performing a morphological operation on the first organ image to obtain a target area specifically includes: Determining a target gap in the first organ image according to a preset gap screening criterion; Determining a corresponding expansion coefficient according to the size of the target gap; Based on the expansion coefficient, a dilation operation is performed on the first organ image to fill at least a portion of gaps in the first organ image.
2. The method according to claim 1, It is characterized in that The step of determining the target gap in the first organ image according to a preset gap screening standard specifically includes: Searching and determining N inflection points along the boundary of the first organ image, where N is a natural number; Screening and determining a target inflection point among the N inflection points, wherein the target inflection point is divided into a concave point and a convex point; The target gap is determined according to the position distribution of the convex points and concave points.
3. The method according to claim 2, It is characterized in that The step of searching and determining N inflection points along the boundary of the first organ image specifically includes: Starting from the preset first node, along the boundary of the first organ image, determining the second to Kth nodes in sequence; Among the K nodes, searching for a node whose boundary direction changes as the inflection point; The boundary direction is determined according to the angle between two adjacent plane vectors; among the two adjacent plane vectors, one plane vector is composed of the k-1th node and the kth node, and the other plane vector is composed of the kth node and the k+1th node.
4. The method according to claim 2, It is characterized in that The screening and determining of the target inflection point among the N inflection points specifically includes: Determining a plurality of target inflection points according to the distances between the N inflection points; According to the relative direction of each target inflection point, the target inflection point is divided into a convex point or a concave point; The target inflection point is a node on the boundary of the organ unit, the distance between which and two adjacent inflection points meets a preset distance standard.
5. The method according to claim 2, It is characterized in that Determining the target gap according to the position distribution of the convex points and the concave points specifically includes: Calculating the Euclidean distance between the concave point and the convex point; According to the Euclidean distance between the concave point and the different convex points, a plurality of candidate gaps are obtained by matching; each of the candidate gaps is composed of at least one concave point and at least two convex points; Calculating the sum of the Euclidean distances of each candidate gap; the sum of the Euclidean distances is: the superposition of the Euclidean distances between the convex points and the concave points; The target gap is determined from the plurality of candidate gaps according to the sum of the Euclidean distances.
6. The method according to claim 2, It is characterized in that Determining the corresponding expansion coefficient according to the size of the target gap specifically includes: calculating a gap width and a gap depth of the gap; When the gap width is greater than the gap depth, determining an expansion coefficient proportional to the gap width; When the gap depth is greater than the gap width, an expansion coefficient is determined that is proportional to the gap depth.
7. The image processing method according to any one of claims 1 to 6, It is characterized in that The performing morphological operation on the first organ image to obtain the target area further comprises: According to the expansion coefficient, determining a corresponding corrosion coefficient; Based on the erosion coefficient, performing an erosion operation on the first organ image after the dilation operation; The corrosion coefficient is greater than the expansion coefficient, and the difference between the corrosion coefficient and the expansion coefficient is within a preset difference range.
8. The method according to claim 1, It is characterized in that The method further comprises: performing image post-processing on the second organ image to obtain an output image of the second organ; The image post-processing includes one or more of the following: expanding the second organ image; removing impurity parts of the second organ image that do not belong to the second organ; The third organ image is removed from the second organ image.
9. The method according to claim 8, It is characterized in that The removing of the impurity portion of the second organ image that does not belong to the second organ specifically includes: Determine a connected area in the second organ image and a circumscribed cube corresponding to each of the connected areas; Calculate the number of pixels in the connected area and the volume of the corresponding circumscribed cube; determining a ratio between the volume and the number of pixels; When the ratio is greater than a preset threshold, the connected area is removed.
10. The method according to claim 8, It is characterized in that The removing of the impurity portion of the second organ image that does not belong to the second organ specifically includes: determining a connected region in the second organ image; Counting the number of pixels in each of the connected regions respectively; When the number of pixels is less than a preset threshold, removing the connected area; Determine the circumscribed cube corresponding to each of the remaining connected regions; Calculating the number of pixels in the remaining connected area and the volume of the corresponding circumscribed cube; determining a ratio between the volume and the number of pixels; When the ratio is greater than a preset threshold, the connected area is removed.
11. The method according to claim 8, It is characterized in that The removing the third organ image from the second organ image specifically includes: Acquire third organ images; performing a dilation operation on the third organ image to obtain an expanded image of the third organ; comparing the expanded image of the third organ and the image of the second organ to determine an overlapping portion therebetween; In the second organ image, the overlapping portion is removed.
12. The method according to claim 11, It is characterized in that The third organ is the trachea; the second organ is the pulmonary blood vessels; The step of performing a dilation operation on the third organ image to obtain an expanded image of the third organ specifically includes: Determine three structural elements for performing expansion operations in three-dimensional directions; the structural elements are ellipsoidal; Based on the structure element, dilation processing is performed on pixels that do not belong to the trachea in sequence to obtain a dilated image of the third organ.
13. An image processing device, It is characterized in that include: at least one processor; and a memory communicatively coupled to the at least one processor; The memory stores computer program instructions, and the computer program instructions are executed by the at least one processor so that the at least one processor can execute the image processing method according to any one of claims 1 to 12.
14. A computer-readable storage medium, It is characterized in that The computer storage medium stores a computer program, and when the computer program is executed, the image processing method according to any one of claims 1 to 12 is implemented.
Citation Information
Patent Citations
Automatic division method for pulmonary parenchyma of CT image
CN104992445A
A lung nodule auxiliary detection system and method based on an improved convex hull method
CN109300113A
Image segmentation method, readable storage medium and image processing device
CN110648338A
Self-adaptive threshold segmentation method and device for lung CT density increase shadow
CN114764809A
Method for Extracting Airways and Pulmonary Lobes and Apparatus Therefor
US20160189373A1
