Intelligent colon polyp edge sketching method based on medical image

Abstract

The invention relates to the technical field of edge segmentation, in particular to an intelligent colon polyp edge sketching method based on medical image.The method comprises the steps that in the colonoscopy process, edge detection is conducted on each frame of image in collected colon internal detection image data, and a plurality of nested intestinal wall contour curves are obtained; determining the normal deviation degree of each position point based on the normal direction change of each position point on the intestinal wall profile curve, and further determining a polyp position point on the intestinal wall profile curve; and based on curvature change and normal direction change of adjacent position points of the polyp position points on the intestinal wall contour curve, determining a polyp edge end point of the polyp position points, and performing colon inner wall polyp contour sketching in each frame of image. The polyp edge recognition precision is effectively improved by accurately positioning the polyp protruding area on the intestinal wall and drawing the edge contour.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of edge segmentation, and in particular to a colon polyp edge intelligent delineation method based on medical images. BACKGROUND

[0002] In the clinical diagnosis and treatment system, the screening and early diagnosis of colon polyps have a milestone significance for the prevention and treatment of colorectal cancer. Medical research has confirmed that about 80% to 95% of colorectal cancer is evolved from adenomatous polyps, and early detection and timely implementation of endoscopic resection can greatly reduce the risk of colon cancer.

[0003] The traditional colon polyp identification method mainly relies on the naked eye observation and experience judgment of endoscopists, and the polyps are identified artificially through the characteristics such as the shape, color and surface texture in the endoscopic images. However, this method has obvious limitations: on the one hand, visual assessment is easily affected by subjective experience, and the identification sensitivity for small or flat polyps is insufficient; on the other hand, when the polyp boundary is fuzzy or the contrast with the surrounding mucosa is low, it is difficult for the naked eye to accurately define the polyp edge, which may lead to missed diagnosis or misjudgment. SUMMARY

[0004] In order to solve the above technical problem of difficult to accurately define the polyp edge, the purpose of the present application is to provide a colon polyp edge intelligent delineation method based on medical images, and the technical solution adopted is as follows: In the first aspect, the present application provides a colon polyp edge intelligent delineation method based on medical images, comprising the following steps: During the colonoscopy process, edge detection is performed on each frame of image in the collected colon internal detection image data to obtain a plurality of nested intestinal wall contour curves; Based on the normal direction change of each position point on the intestinal wall contour curve, the normal deviation degree of each position point on the intestinal wall contour curve is determined; Based on the normal deviation degree, the polyp position point on the intestinal wall contour curve is determined; Based on the curvature change and normal direction change of the adjacent position points of the polyp position point on the intestinal wall contour curve, the polyp edge end point of the polyp position point is determined; Based on the polyp position point and its polyp edge end point, colon inner wall polyp contour delineation is performed in each frame of image; Determining the normal deviation degree of each position point on the intestinal wall contour curve comprises: Taking the center point of the innermost intestinal wall contour curve as the starting point, a plurality of radial lines are constructed; Determining the intersection point of each radial line and the plurality of intestinal wall contour curves; determine a normal deviation degree of the intersection point on the intestinal wall profile curve based on a normal direction difference between the intersection point and other position points on the intestinal wall profile curve.

[0005] In some possible implementation manners of the first aspect, the method further includes: performing curve fitting on the intestinal wall profile curve to obtain a fitting equation; determine normal position points according to a condition that each position point on the intestinal wall profile curve meets the fitting equation; determine an average value and a standard deviation of angles corresponding to normal directions of all the normal position points on the intestinal wall profile curve to obtain an angle average value and an angle standard deviation; determine an angle difference value of a difference between an angle corresponding to a normal direction of the intersection point on the intestinal wall profile curve and the angle average value; determine a ratio of the angle difference value and the angle standard deviation to obtain the normal deviation degree of the intersection point on the intestinal wall profile curve.

[0006] In some possible implementation manners of the first aspect, the method further includes: for any one position point on two sides of the polyp position point on the intestinal wall profile curve, determine a polyp edge end point score of the any one position point based on a curvature difference and a normal direction difference between the any one position point and a neighboring position point on a side close to the polyp position point, and a curvature difference and a normal direction difference of each position point in a neighboring window region of the any one position point; determine a target neighboring position point closest to the polyp position point from each side neighboring position point of the polyp position point on the intestinal wall profile curve as a polyp edge end point, where a polyp edge end point score of the target neighboring position point is greater than a score threshold.

[0007] In some possible implementation manners of the first aspect, the method further includes: determine a curvature difference value and an angle difference value of an angle corresponding to a normal direction between the any one position point and the neighboring position point on the side close to the polyp position point; determine an average value of the curvature difference of each position point in the neighboring window region of the any one position point to obtain an average curvature difference; determine an average value of the angle difference value of each position point in the neighboring window region of the any one position point to obtain an average angle difference; determine a first ratio value of the curvature difference value and the average curvature difference; determining a ratio of the angle difference value to the average angle difference to obtain a second ratio; performing weighted addition on the first ratio and the second ratio by using a dynamic weight coefficient to obtain a polyp edge endpoint score of the arbitrary position point.

[0008] In combination with the first aspect, in some possible implementation manners, the determination process of the dynamic weight coefficient comprises: judging a relationship between the curvature of the arbitrary position point on the intestinal wall contour curve and the average curvature difference; if it is judged that the curvature of the arbitrary position point is greater than a first set value multiple of the average curvature difference, setting the dynamic weight coefficient of the first ratio to be greater than the dynamic weight coefficient of the second ratio, and the first set value is greater than 1; if it is judged that the curvature of the arbitrary position point is less than a second set value multiple of the average curvature difference, setting the dynamic weight coefficient of the first ratio to be less than the dynamic weight coefficient of the second ratio, and the second set value is less than 1; otherwise, setting the dynamic weight coefficient of the first ratio to be equal to the dynamic weight coefficient of the second ratio.

[0009] In combination with the first aspect, in some possible implementation manners, the determination process of the score threshold value comprises: determining a distance from each of the adjacent position points on each side to the polyp position point; adjusting a set basic threshold value based on the distance to obtain an adjusted threshold value; determining a minimum value of a set upper limit threshold value and the adjusted threshold value; taking the minimum value as the polyp position point on the intestinal wall contour curve.

[0010] In combination with the first aspect, in some possible implementation manners, after the colon inner wall polyp contour is outlined in each frame of image, the method further comprises: for the same polyp position point, determining a contour length of the colon inner wall polyp contour outlined in the current frame of image and all previous frames of image; based on the contour length in all previous frames of image, predicting the contour length in the current frame of image to obtain an expected contour length; based on a deviation amount between the contour length in the current frame of image and the expected contour length, judging whether the colon inner wall polyp contour in the current frame of image is outlined completely; if not, adjusting the colonoscope angle to obtain a new colon inner detection image to re-outline the colon inner wall polyp contour.

[0011] In conjunction with the first aspect above, in some possible implementations, determining whether the outline of the colon wall polyp in the current frame image is complete includes: Determining a deviation threshold based on a standard deviation of the contour lengths in all frame images before the current frame image; If the deviation is greater than the deviation threshold, it is determined that the outline of the colon inner wall polyp in the current frame image is incomplete; otherwise, it is determined that the outline of the colon inner wall polyp in the current frame image is complete.

[0012] In combination with the first aspect above, in some possible implementations, the method further includes: Based on the outline of the colonic polyp, the colonic polyps were classified.

[0013] In a second aspect, the present invention further provides a device for intelligently delineating the edges of colon polyps based on medical images, the device comprising: The first module is used to perform edge detection on each frame of the collected colon internal detection image data during the colonoscopy process to obtain a plurality of nested intestinal wall contour curves; The second module is used to determine the degree of normal deviation of each position point on the intestinal wall contour curve based on the change of the normal direction of each position point on the intestinal wall contour curve; A third module is configured to determine a polyp location point on the intestinal wall contour curve based on the normal deviation degree; A fourth module is configured to determine an end point of a polyp edge at the polyp location point based on a change in curvature and a change in normal direction of adjacent locations of the polyp location point on the intestinal wall contour curve; A fifth module is configured to delineate the outline of the polyp on the inner wall of the colon in each frame of image based on the polyp location point and the polyp edge end point; Determining the normal deviation degree of each position point on the intestinal wall contour curve includes: Taking the center point of the innermost intestinal wall contour curve as the starting point, construct several radial lines; determining an intersection point between each of the radial lines and the plurality of intestinal wall contour curves; Based on the difference in normal direction between the intersection point and other position points on the intestinal wall contour curve, the degree of normal deviation of the position point where the intersection point is located on the intestinal wall contour curve is determined.

[0014] In a third aspect, the present invention further provides a medical imaging-based intelligent delineation system for colon polyp margins, comprising a memory and a processor. The memory is configured to store executable computer program code, and the processor is configured to retrieve and execute the executable computer program code from the memory, causing the system to perform the method of the first aspect or any possible implementation of the first aspect.

[0015] In a fourth aspect, the present application also provides a computer program product comprising computer program code which, when executed on a computer, causes the computer to perform the method of the first aspect or any possible implementation mode of the first aspect.

[0016] In a fifth aspect, the present application also provides a computer-readable storage medium storing computer program code which, when executed on a computer, causes the computer to perform the method of the first aspect or any possible implementation mode of the first aspect.

[0017] The present application has the following beneficial effects: in the process of colonoscopy, by acquiring the intestinal wall contour curve of each frame of image in the colon internal detection image data, and based on the normal direction change of each position point on the intestinal wall contour curve, the polyp position point on the intestinal wall contour curve is accurately identified; meanwhile, based on the curvature change and the normal direction change of the adjacent position points of the polyp position point on the intestinal wall contour curve, the polyp edge end point of the polyp position point is accurately identified; finally, based on the polyp position point and the polyp edge end point, the complete outlining of the colon inner wall polyp contour is performed. The present application accurately positions the polyp position point and the polyp edge end point on the intestinal wall and performs the complete outlining of the edge contour, thereby effectively improving the polyp edge recognition accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, and the advantages thereof, the drawings needed to be used in the embodiments or prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor based on these drawings.

[0019] Figure 1 A step flow chart of a medical image-based colon polyp edge intelligent outlining method according to an embodiment of the present application; Figure 2 A schematic diagram of a plurality of intestinal wall contour curves constructed in a certain frame of image according to an embodiment of the present application; Figure 3 A schematic diagram of the normal direction of the intestinal wall contour curve and the polyp edge at a certain polyp position point according to an embodiment of the present application; Figure 4 A structural schematic diagram of a medical image-based colon polyp edge intelligent outlining system according to an embodiment of the present application. DETAILED DESCRIPTION

[0020] In order to clearly illustrate the technical features of the present application, the present application will be described in detail below by means of specific implementation modes and in combination with the drawings.

[0021] Embodiments of the present application will be described in more detail with reference to the drawings. While several embodiments of the application are shown in the drawings, it is not intended that the application be limited to the embodiments shown, and it is to be understood that the application can be carried out by way of various forms.

[0022] It should be understood that each of the steps in the method embodiments of the present application can be performed in a different order and / or in parallel. In addition, the method embodiments can include additional steps and / or omit performing the steps shown. The scope of the present application is not limited in this respect.

[0023] The term "comprising" and variations thereof as used herein are used inclusively, i.e., "comprising but not limited to." The term "based on" is "based at least in part on." The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments." Related terms are defined in the description that follows.

[0024] It should be noted that the terms "first", "second", and so on used in the present application are merely used to distinguish different devices, modules or units, and are not intended to limit the order or the interdependence of the functions performed by these devices, modules or units.

[0025] In the embodiments of the present application, although the operations or steps are described in a particular order in the accompanying drawings, it should not be understood as requiring the operations or steps to be performed in the particular order or in a serial order, or requiring all of the operations or steps to be performed to obtain a desired result. In the embodiments of the present application, the operations or steps can be performed in series; the operations or steps can be performed in parallel; or a part of the operations or steps can be performed.

[0026] At the same time, it can be understood that the data involved in the technical solutions of the present application (including but not limited to the data itself, the acquisition or use of the data) should comply with the requirements of the corresponding laws, regulations and relevant provisions. Unless otherwise defined, all technical and scientific terms used in the present application have the same meaning as understood by a person skilled in the art to which the present application belongs, and all parameters or indexes in the formulas involved in the present application are normalized values that eliminate the influence of dimensions.

[0027] In order to solve the problem that it is difficult to accurately define the edge of the polyp, the embodiment of the present application provides a colon polyp edge intelligent sketching method based on medical images. The method analyzes the colon internal detection image data shot by a colonoscope, identifies a plurality of intestinal wall contour curves embedded in each frame of image, accurately locates the polyp position point and the polyp edge end on the intestinal wall, and performs complete edge contour sketching, thereby effectively improving the polyp edge recognition accuracy and providing a reliable basis for doctors to judge the colon condition of patients.

[0028] The colon polyp edge intelligent sketching method based on medical images provided by the embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0029] Figure 1 The basic flowchart of the colon polyp edge intelligent sketching method based on medical images provided by the embodiment of the present application is shown in FIG. 1. Figure 1 As shown in FIG. 1, the method specifically comprises the following steps: Step S100: In the process of colonoscopy, edge detection is performed on each frame of image in the acquired colon internal detection image data, and a plurality of nested intestinal wall contour curves are obtained.

[0030] Specifically, in order to realize accurate sketching of the colon polyp edge, a standardized process is adopted to obtain high-quality colon internal detection image data of the patient, that is, the patient is adjusted to a low residue diet for 3 days before examination, and bowel preparation is completed 1 day before examination. In the process of colonoscopy, the high-definition camera of the colonoscope is used to dynamically collect the colon internal intestinal mucosa image at a speed of 30 frames per second, and image enhancement techniques such as narrowband imaging are used to enhance the collected image to improve the image contrast. The detection image after enhancement is stored in the medical image system in real time, which provides continuous frame colon internal detection image data for subsequent polyp edge sketching.

[0031] In the process of colonoscopy, edge detection is performed on each frame of image in the acquired colon internal detection image data, such as using Canny edge detection algorithm for edge detection, and a plurality of continuous and nested intestinal wall contour curves in each frame of image are constructed. Figure 2 A plurality of intestinal wall contour curves constructed in a frame of image are shown in FIG. 2. Subsequently, by analyzing the change characteristics of the plurality of intestinal wall contour curves constructed in each frame of image, the intestinal wall polyp region is accurately positioned and the edge is sketched.

[0032] Step S200: Based on the normal direction change of each position point on the intestinal wall contour curve, the normal deviation degree of each position point on the intestinal wall contour curve is determined.

[0033] Specifically, in normal cases, the inner wall of a healthy colon is smooth, and its structure appears as a series of nested circular contours in a two-dimensional image, which are presented in a form of one inside the other, reflecting the hierarchical structure of the intestinal wall. However, in a colonoscopy image in which a polyp may occur, the regular contours may be locally deformed, causing the normal direction of the local contour to change. Therefore, for each frame of image data detected in the colon interior, the change in the normal direction of each position point on the intestinal wall contour curve in the image is analyzed to determine the normal deviation degree, so as to quantify the degree of deviation of the normal direction of the position point from the normal case, thereby accurately identifying the position point where the polyp occurs.

[0034] Further, in a possible implementation, the normal deviation degree of each position point on the intestinal wall contour curve is determined, including: Step S201: taking the center point of the innermost intestinal wall contour curve as a starting point, a plurality of radial lines are constructed.

[0035] Specifically, in order to more accurately analyze the normal deviation of each position point on the intestinal wall contour curve, a plurality of radial lines are constructed with the center point of the innermost intestinal wall contour curve as a starting point. In a specific example, a polar coordinate system is established with the center point of the innermost intestinal wall contour curve as an origin and the horizontal right as a polar axis. Starting from the polar axis, rotate in a clockwise direction, and draw a radial line along the polar axis every time a set angle is rotated, thereby constructing a plurality of radial lines. The size of the set angle can be reasonably set according to needs to balance the detection accuracy and the amount of calculation.

[0036] In a specific example, the set angle is determined by the following formula: ; In the formula: represents a basic angle interval (by default ); represents the average radius of curvature of all intestinal wall contour curves in each frame of image; represents the average radius of curvature of a normal intestinal wall.

[0037] In the above formula, when the local radius of curvature of the intestinal wall in the image is small, the angle is reduced by the formula, so that the sampling points are encrypted in the image to capture more subtle contour deformations; on the contrary, when the radius of curvature of the intestinal wall is close to the normal value, a larger angle is maintained to reduce calculation redundancy.

[0038] Step S202: determining the intersection points of each radial line and a plurality of intestinal wall contour curves.

[0039] Specifically, the radiation lines will pass through each intestinal wall contour curve in turn and generate intersection points with the same. Assuming that the inner wall of the colon appears as I nested intestinal wall contour curves in a frame image, the intersection set of the nth radiation line and the I nested intestinal wall contour curves is denoted as wherein, represents the intersection of the nth radiation line and the i-th intestinal wall contour curve.

[0040] Step S203: determining the normal deviation degree of the position point of the intersection on the intestinal wall contour curve based on the normal direction difference between the intersection and other position points on the intestinal wall contour curve.

[0041] Specifically, since the normal intestinal wall is a circular contour, the normal direction of the contour is concentrated in the elliptical statistical reference range (determined by the mean value and the standard deviation), and the polyp region deviates from the reference range due to the sharp change in local curvature. Therefore, the normal direction difference between the intersection and other position points on the intestinal wall contour curve is analyzed to determine the normal deviation degree of the position point of each intersection, so as to evaluate the deviation degree of the normal direction of each position point of the intersection from the normal case.

[0042] In a specific example, the normal deviation degree of the position point of the intersection on the intestinal wall contour curve is determined, including: performing curve fitting on the intestinal wall contour curve to obtain a fitting equation; determining normal position points according to the fitting equation of each position point on the intestinal wall contour curve; determining the mean value and the standard deviation of the angles corresponding to the normal directions of all normal position points on the intestinal wall contour curve to obtain the angle mean value and the angle standard deviation; determining the absolute value of the difference between the angle corresponding to the normal direction of the intersection and the angle mean value to obtain the angle difference value; and determining the ratio of the angle difference value to the angle standard deviation to obtain the normal deviation degree of the position point of the intersection on the intestinal wall contour curve.

[0043] In this specific example, since the polyp is a local abnormal protruding lesion, it only occupies a very small part of the intestinal wall region, and therefore the normal region can be segmented by global geometric features, that is, an coordinate system is constructed, and an ellipse fitting is performed on each intestinal wall contour curve to obtain a fitting equation: wherein, represents the center coordinates of the fitted ellipse, and are the half-axis lengths of the fitted ellipse on the x-axis and the y-axis, respectively. The coordinates of each position point on each intestinal wall contour curve are substituted into the fitting equation, and if wherein, represents the tolerance for controlling inward recess, and is set as , represents the tolerance for controlling outward protrusion, and is set as , then the corresponding position point is determined to be a normal position point, thereby determining each normal position point in each intestinal wall contour curve.

[0044] For the intersection set Any intersection point in The degree of normal deviation of each intersection point on the intestinal wall contour curve is determined by the following calculation formula: ; Where, Indicates the intersection point of the intestinal wall contour curve The degree of deviation of the normal line of the location point; Indicates the intersection point of the intestinal wall contour curve The angle corresponding to the normal direction of the location point; They represent the angle mean and angle standard deviation, that is, the average value and standard deviation of the angles corresponding to the normal direction of all normal position points on several intestinal wall contour curves.

[0045] Step S300: Determine the polyp location point on the intestinal wall contour curve based on the degree of normal deviation.

[0046] Specifically, when the value of the normal deviation is greater, it means that the position point at the corresponding intersection is more likely to be an abnormal point in the polyp area. Therefore, based on the normal deviation, the polyp position point on the intestinal wall contour curve can be identified. In a specific example, a deviation threshold T is set in advance, such as setting T to 2 or 3, and the normal deviation of each intersection in the intersection set on the intestinal wall contour curve is compared with the set deviation threshold T. When the normal deviation is greater than the set deviation threshold T, the position point at the corresponding intersection on the intestinal wall contour curve is determined as a polyp position point. In this way, by traversing any intersection in the intersection set corresponding to each radial line, abnormal points that exceed the statistical benchmark range are screened out. These abnormal points are the position points in the polyp area, and the edges of the colon polyp area can be subsequently outlined based on these points.

[0047] Step S400: determining the polyp edge endpoint of the polyp location point based on the curvature change and normal direction change of adjacent location points of the polyp location point on the intestinal wall contour curve.

[0048] Specifically, after locating the intersection of the polyp region, the algorithm uses that point as the starting point to traverse the pixels in both the left and right directions along the intestinal wall contour curve, recording the normal direction and curvature value one by one. When the traversal point is at the edge of the polyp, its normal direction points to the inside of the polyp, and the curvature value may be higher or lower than the inner wall of the quasi-circular wall depending on the diversity of the polyp morphology. When the traversal point reaches the inner wall of the quasi-circular wall, the normal direction points to the center of the colon wall, and the curvature value remains constant at the quasi-circular curvature. Figure 3A normal direction diagram of the intestinal wall profile curve and the polyp edge at a polyp position point is shown.

[0049] Therefore, the polyp edge end point can be extracted by analyzing the difference between the two types of features to construct the polyp profile. Since a single feature (such as only curvature or normal direction) cannot cover all polyp morphologies in the case of unknown polyp morphology, the applicability can be improved by complementary features, and the joint determination of curvature and normal direction is realized to accurately identify the polyp edge end point of the polyp position point on the intestinal wall profile curve.

[0050] Further, in a possible implementation, the polyp edge end point of the polyp position point is determined, including: Step S401: For any one position point on both sides of the polyp position point on the intestinal wall profile curve, based on the curvature difference and the normal direction difference between the any one position point and the adjacent position point on the side close to the polyp position point, and the curvature difference and the normal direction difference of each position point in the adjacent window region of the any one position point, the polyp edge end point score of the any one position point is determined.

[0051] Specifically, starting from the polyp position point on the intestinal wall profile curve, the pixel points are traversed in both left and right directions along the intestinal wall profile curve. In the traversal process, compared with the surrounding position points, when the curvature and the normal direction of a certain position point change greatly relative to the curvature and the normal direction of the previous position point, it means that the position point is most likely to be the polyp edge end point. Therefore, by comparing the curvature difference and the normal direction difference between each position point and its previous position point with the curvature difference and the normal direction difference of the surrounding position points, the polyp edge end point score of each position point is obtained, and the polyp edge end point is accurately identified based on the polyp edge end point score.

[0052] In a specific example, the polyp edge end point score of the any one position point is determined, including: determining the curvature difference value and the angle difference value of the corresponding angle of the normal direction between the any one position point and the adjacent position point on the side close to the polyp position point; determining the average value of the curvature difference of each position point in the adjacent window region of the any one position point to obtain the average curvature difference; determining the average value of the angle difference value of each position point in the adjacent window region of the any one position point to obtain the average angle difference; determining the ratio of the curvature difference value to the average curvature difference to obtain a first ratio value; determining the ratio of the angle difference value to the average angle difference to obtain a second ratio value; and weighting and adding the first ratio value and the second ratio value by using a dynamic weight coefficient to obtain the polyp edge end point score of the any one position point.

[0053] In this specific example, the polyp edge end point score of the any one position point on both sides of the polyp position point on the intestinal wall profile curve is calculated by the following calculation formula: ; In the formula: represents the polyp edge end point score of an arbitrary position point on the two sides of the polyp position point on the intestinal wall contour curve; represents the curvature difference value between the arbitrary position point on the two sides and the adjacent position point on the side close to the polyp position point, that is, the absolute value of the curvature difference between the arbitrary position point on the two sides and the previous adjacent position point in the traversal process; represents the angle difference value of the normal direction corresponding angle between the arbitrary position point on the two sides and the adjacent position point on the side close to the polyp position point, that is, the absolute value of the difference between the normal direction corresponding angle of the arbitrary position point on the two sides and the previous adjacent position point in the traversal process; represents the average value of the curvature difference of each position point in the adjacent window region (such as a 5-pixel window centered on the arbitrary position point) of the arbitrary position point on the two sides, that is, the average curvature difference of each position point in the adjacent window region of the arbitrary position point on the two sides; represents the average value of the angle difference value of each position point in the adjacent window region (such as a 5-pixel window centered on the arbitrary position point) of the arbitrary position point on the two sides, that is, the average angle difference of each position point in the adjacent window region of the arbitrary position point on the two sides; 、 represents a dynamic weight coefficient, and α+β=1.

[0054] In the above formula, the dynamic weight coefficient 、 adjusts the proportion of the influence of the curvature and the normal direction to accurately determine the polyp edge end point score of the arbitrary position point on the two sides of the polyp position point on the intestinal wall contour curve. The specific value of the dynamic weight coefficient 、 is determined according to the relationship between the curvature of the arbitrary position point on the two sides of the polyp position point on the intestinal wall contour curve and the average curvature difference . Since the junction end point of the polyp and the normal intestinal wall is also the polyp edge end point, which is the key position of the transition of the polyp shape from abnormal to normal, the morphological mutation characteristics of the edge end point are significant, resulting in the increase of the curvature change parameters and in the above formula, which shows that the score is higher.

[0055] Further, in a possible implementation, the dynamic weight coefficient 、 The determination process of the first ratio includes: judging the relationship between the curvature of any position point on the intestinal wall contour curve and the average curvature difference; if the curvature of any position point is greater than the first set value times of the average curvature difference, setting the dynamic weight coefficient of the first ratio to be greater than the dynamic weight coefficient of the second ratio, and the first set value is greater than 1; if the curvature of any position point is less than the second set value times of the average curvature difference, setting the dynamic weight coefficient of the first ratio to be less than the dynamic weight coefficient of the second ratio, and the second set value is less than 1; otherwise, setting the dynamic weight coefficient of the first ratio to be equal to the dynamic weight coefficient of the second ratio.

[0056] In a specific example, the first set value is set to 2, and the second set value is set to 0.5. When the curvature of any position point is greater than 2 times of the average curvature difference, the dynamic weight coefficient of the first ratio is set to be greater than the dynamic weight coefficient of the second ratio. When the curvature of any position point is less than 0.5 times of the average curvature difference, the dynamic weight coefficient of the first ratio is set to be less than the dynamic weight coefficient of the second ratio. Otherwise, the dynamic weight coefficient of the first ratio is set to be equal to the dynamic weight coefficient of the second ratio.

[0057] Step S402: determining the target adjacent position point closest to the polyp position point in the polyp edge end point score greater than the score threshold value of each side adjacent position point of the polyp position point on the intestinal wall contour curve as the polyp edge end point.

[0058] Specifically, in the process of traversing the surrounding pixel points along the intestinal wall contour curve in the left and right directions from the polyp position point as the starting point, the polyp edge end point score of each position point on both sides of the polyp position point can be obtained in turn. When the polyp edge end point score is greater than the score threshold value, the corresponding position point is taken as the target adjacent position point, and the target adjacent position point is the polyp edge end point of the polyp position point.

[0059] In the process of traversing the surrounding pixel points along the intestinal wall contour curve in the left and right directions of the polyp position point, as the traversal distance increases, the traversed pixel points gradually move away from the starting point. However, the long-distance traversal is easily disturbed by noise, which may lead to misjudgment of the noise as the edge end point. Therefore, as the traversal distance increases, the score threshold value is dynamically improved to avoid misjudgment of the pixel points as the polyp edge end point in these long-distance regions.

[0060] Further, in a possible implementation, the determination process of the score threshold value includes: determining the distance from each adjacent position point in the adjacent position points on both sides of the polyp position point to the polyp position point; adjusting the set basic threshold value based on the distance to obtain an adjusted threshold value; determining the minimum value of the set upper limit threshold value and the adjusted threshold value; and taking the minimum value as each adjacent position point in the adjacent position points on both sides of the polyp position point on the intestinal wall contour curve.​​​​​​

[0061] In a specific example, the score threshold is calculated by the following formula: ; In the formula: represents a set upper threshold, which is used to prevent extreme noise or feature fluctuations from causing the score threshold to grow indefinitely, ensuring stability, and is usually set to 1.5; represents a set base threshold, as the polyp position point may deviate from the true boundary due to segmentation errors or noise, and is set to be relatively low, such as 0.8, to reduce the risk of missed detection; represents a distance decay coefficient, which needs to be calibrated in combination with actual data, and is usually taken as 0.1-0.4; represents the distance from each of the adjacent position points on each side of the intestinal wall contour curve to the polyp position point, i.e., the number of pixel points from each of the adjacent position points on each side of the intestinal wall contour curve to the polyp position point.

[0062] In the above manner, by determining the polyp edge end point score of any position point on either side of the polyp position point on the intestinal wall contour curve, and comparing the polyp edge end point score with the adaptively determined score threshold, the polyp edge end points on both sides of the polyp position point on the intestinal wall contour curve can be accurately identified.

[0063] Step S500: Based on the polyp position point and its polyp edge end points, the colon inner wall polyp contour is outlined in each frame of image.

[0064] Specifically, starting from the polyp position point on the intestinal wall contour curve, the surrounding pixel points are traversed and tracked in both left and right directions along the intestinal wall contour curve until the polyp edge end points are reached, thereby realizing the contour outlining of the colon inner wall polyp region.

[0065] When the colonoscope is pushed forward in the colon, if the colon inner wall polyp contour is complete, its contour should show a gradual enlargement in consecutive multiple frames of image. However, when the polyp is partially blocked by intestinal wall folds and other structures, the contour will suddenly increase during the movement of the colonoscope. Based on this dynamic change characteristic, the integrity of the outlined colon inner wall polyp contour needs to be verified.

[0066] Further, after the colon inner wall polyp contour is outlined in each frame of image, the method further comprises: Step S501: For the same polyp position point, the contour length of the colon inner wall polyp contour outlined in the current frame of image and all previous frames of image is determined.

[0067] Specifically, the contour length of the colon inner wall polyp contour outlined in each frame of image after the polyp appears is counted.

[0068] Step S502: Based on the contour lengths in all frame images before the current frame image, the contour length in the current frame image is predicted to obtain an expected contour length.

[0069] Specifically, for the same polyp location point, based on the changes in contour length in all frame images before the current frame image, a normal contour length change pattern is established as a reference benchmark for subsequent detection.

[0070] In a specific example, for the same polyp location point, the contour length growth rate is calculated using the following formula based on the contour lengths in all frames before the current frame: ; Where: represents the contour length growth rate; L(N) represents the contour length in the Nth frame image before the current frame image; L(1) represents the contour length in the 1st frame image before the current frame image; N represents the number of image frames before the current frame image after the polyp location point is detected.

[0071] Furthermore, for the same polyp location point, based on the contour length in the previous frame image of the current frame image and contour length growth rate , determine the expected contour length in the current frame image .

[0072] Step S503: Based on the deviation between the contour length in the current frame image and the expected contour length, it is determined whether the contour of the colon inner wall polyp in the current frame image is completely drawn.

[0073] Specifically, the ratio of the absolute value of the difference between the contour length in the current frame image and the expected contour length to the expected contour length is determined to obtain a deviation. If the deviation between the contour length of the colonic polyp outlined in the current frame image and the expected contour length is small, it indicates that the contour of the colonic polyp in the current frame image is complete; otherwise, it indicates that the contour is complete.

[0074] Furthermore, in a possible implementation, determining whether the outline of the colon polyp in the current frame image is complete includes: determining a deviation threshold based on the standard deviation of the outline lengths in all frame images before the current frame image; if the deviation is greater than the deviation threshold, determining that the outline of the colon polyp in the current frame image is incomplete; otherwise, determining that the outline of the colon polyp in the current frame image is complete. In a specific example, determining the standard deviation of the outline lengths in all frame images before the current frame image , get the deviation threshold , when the deviation is greater than the deviation threshold If yes, it is determined that the contouring is abnormal, i.e., the length of the contour changes suddenly, and the contouring is determined to be abnormal, otherwise, the contouring is determined to be normal.

[0075] Step S504: If the contouring is incomplete, the colonoscope angle is adjusted to obtain a new colon interior detection image to re-contour the polyp on the colon inner wall.

[0076] Specifically, if it is determined that the polyp contouring is incomplete, the colonoscope angle needs to be adjusted to re-contour the polyp on the colon inner wall by re-obtaining a colon interior detection image.

[0077] After the polyp contouring on the colon inner wall is completed, the polyp on the colon inner wall is classified based on the contoured polyp contour. In a specific example, the classification process includes: according to the ratio R / L of the curvature radius R of the polyp contour under the colonoscope to the base length L (i.e., the distance between the two points where the polyp contour connects with the intestinal wall contour), the polyp can be divided into three categories: the protruding type (R / L≥0.5), the base is wide and the surface is hemispherical protrusion, which is often seen in adenomas with a diameter >1 cm; the sub-ty type (0.3≤R / L<0.5), the neck forms obvious narrowing but is not completely free, often accompanied by surface villous structure; the long-ty type (R / L<0.3), the length of the peduncle is more than 2 times the diameter of the lesion, the blood vessel runs in a spiral shape, and is easy to twist and bleed.

[0078] In addition, according to the number combination of each type of polyp in a single examination, risk stratification can be performed: low-risk group (1-2): if all are <5 mm protruding type tubular adenoma, 5-year cancer risk <1%, 3-year follow-up is recommended; medium-risk group (3-4): if it contains ≥1 sub-ty type villous adenoma, it needs to be combined with Ki-67 proliferation index, and positive ones need to be rechecked within 1 year; high-risk group (≥5 or any 1 >2 cm): it suggests that there may be hereditary polyposis, and APC gene detection and total colectomy planning are recommended.

[0079] Based on the same inventive concept, the embodiments of the present application also provide a medical image-based colon polyp edge intelligent contouring device, which comprises: A first module is configured to perform edge detection on each frame of image in the collected colon interior detection image data during the colonoscopy process to obtain a plurality of nested intestinal wall contour curves. A second module is configured to determine the normal line deviation degree of each position point on the intestinal wall contour curve based on the normal line direction change of each position point on the intestinal wall contour curve. A third module is configured to determine the polyp position point on the intestinal wall contour curve based on the normal line deviation degree. A fourth module is configured to determine the polyp edge end point of the polyp position point based on the curvature change and the normal line direction change of the adjacent position points of the polyp position point on the intestinal wall contour curve. A fifth module is configured to perform polyp contouring on the inner wall of the colon in each frame of image based on the polyp position point and the polyp edge end point; determine the normal deviation degree of each position point on the intestinal wall contour curve, comprising: taking the center point of the innermost intestinal wall contour curve as a starting point, a plurality of radial lines are constructed; determine the intersection of each of the radial lines and the plurality of intestinal wall contour curves; based on the difference in the normal direction between the intersection point and other position points on the intestinal wall contour curve, determine the normal deviation degree of the position point where the intersection point is located on the intestinal wall contour curve.

[0080] It should be noted that: the device provided by the above embodiment is only exemplified by the division of the above functional modules. In actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the computer device is divided into different functional modules to complete all or part of the functions described above.

[0081] Based on the same inventive concept, the embodiments of the present application also provide a colon polyp edge intelligent contouring system based on medical images, as shown in the accompanying drawings, the system comprises: a memory 401, a processor 402, and a computer program code 403 stored in the memory 401 and running on the processor 402, wherein when the processor 402 executes the computer program code 403, the system can execute any one of the above-mentioned colon polyp edge intelligent contouring methods based on medical images. Figure 4

[0082] The embodiments of the present application can divide the system into functional modules according to the above method examples, for example, each functional module can be corresponding, or two or more functions can be integrated in one processing module, and the above integrated module can be realized in the form of hardware. It should be noted that the division of modules in this embodiment is illustrative, and is only a logical function division. In actual implementation, there can be another division method.

[0083] Based on the same inventive concept, the embodiments of the present application also provide a computer program product, which comprises: computer program code, when the computer program code runs on a computer, the computer executes any one of the above-mentioned colon polyp edge intelligent contouring methods based on medical images.

[0084] Based on the same inventive concept, the embodiments of the present application also provide a computer readable storage medium, which stores computer program code, when the computer program code runs on a computer, the computer executes any one of the above-mentioned colon polyp edge intelligent contouring methods based on medical images.​

[0085] It should be noted that the above-described embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalent ones; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A medical image-based intelligent delineation method for the edges of colon polyps, characterized by, The method comprises the following steps: During a colonoscopy, edge detection is performed on each frame of image data collected from inside the colon to obtain a plurality of nested intestinal wall profile curves; Based on the normal direction changes of each position point on the intestinal wall profile curve, the normal deviation degree of each position point on the intestinal wall profile curve is determined; Based on the normal deviation degree, a polyp position point on the intestinal wall profile curve is determined; Based on the curvature changes and normal direction changes of adjacent position points of the polyp position point on the intestinal wall profile curve, a polyp edge end point of the polyp position point is determined; Based on the polyp position point and the polyp edge end point thereof, a colon inner wall polyp contour is outlined in each frame of image; The determination of the normal deviation degree of each position point on the intestinal wall profile curve comprises: A center point of the innermost intestinal wall profile curve is taken as a starting point to construct a plurality of radial lines; The intersection points of each radial line and the plurality of intestinal wall profile curves are determined; Based on the normal direction differences between the intersection points and other position points on the intestinal wall profile curve, the normal deviation degree of the position point where the intersection point is located on the intestinal wall profile curve is determined.

2. The medical image based intelligent delineation of colonic polyp margin method according to claim 1, wherein, The determination of the normal deviation degree of the position point where the intersection point is located on the intestinal wall profile curve comprises: Curve fitting is performed on the intestinal wall profile curve to obtain a fitting equation; Based on the fitting equation, normal position points are determined according to the fitting equation to which each position point on the intestinal wall profile curve conforms; The average value and the standard deviation of the angles corresponding to the normal directions of all normal position points on the intestinal wall profile curve are determined to obtain an angle average value and an angle standard deviation; The absolute value of the difference between the angle corresponding to the normal direction of the intersection point on the intestinal wall profile curve and the angle average value is determined to obtain an angle difference value; The ratio of the angle difference value to the angle standard deviation is determined to obtain the normal deviation degree of the position point where the intersection point is located on the intestinal wall profile curve.

3. The method for intelligent delineation of colon polyp margins based on medical imaging according to claim 1, characterized in that: The determination of the polyp edge end point of the polyp position point comprises: For any one position point on either side of the polyp position point on the intestinal wall profile curve, based on the curvature difference and the normal direction difference between the any one position point and its adjacent position point on the side close to the polyp position point, and the curvature difference and the normal direction difference of each position point in the adjacent window region of the any one position point, a polyp edge end point score of the any one position point is determined; The target adjacent position point closest to the polyp position point among the adjacent position points on each side of the polyp position point on the intestinal wall profile curve and having a polyp edge end point score greater than a score threshold value is determined as a polyp edge end point.

4. The medical image based intelligent delineation of colonic polyp margin method of claim 3, wherein, The determination of the polyp edge end point score of the any one position point comprises: The curvature difference value and the angle difference value of the angle corresponding to the normal direction between the any one position point and its adjacent position point on the side close to the polyp position point are determined; The average value of the curvature differences of each position point in the adjacent window region of the any one position point is determined to obtain an average curvature difference; The average value of the angle difference values of each position point in the adjacent window region of the any one position point is determined to obtain an average angle difference. determining a ratio of the curvature difference value to the average curvature difference to obtain a first ratio; determining a ratio of the angle difference value to the average angle difference to obtain a second ratio; The first ratio and the second ratio are weightedly added using a dynamic weight coefficient to obtain a polyp edge endpoint score for the arbitrary position point.

5. The medical image based intelligent delineation of colonic polyp margin method of claim 4, wherein, The process of determining the dynamic weight coefficient includes: Determining the relationship between the curvature of any point on the intestinal wall contour curve and the average curvature difference; If it is determined that the curvature of any one of the position points is greater than a first set value multiple of the average curvature difference, setting the dynamic weight coefficient of the first ratio to be greater than the dynamic weight coefficient of the second ratio, and the first set value is greater than 1; If it is determined that the curvature of any one of the position points is less than a multiple of a second set value of the average curvature difference, setting the dynamic weight coefficient of the first ratio to be less than the dynamic weight coefficient of the second ratio, and the second set value is less than 1; Otherwise, the dynamic weight coefficient of the first ratio is set equal to the dynamic weight coefficient of the second ratio.

6. The medical image based intelligent delineation of colonic polyp margin method of claim 3, wherein, The process of determining the score threshold includes: Determine the distance from each adjacent position point on each side of the intestinal wall contour curve to the polyp position point; Adjusting the set basic threshold based on the distance to obtain an adjusted threshold; Determine a minimum value between the set upper threshold and the adjustment threshold; The minimum value is used as each adjacent position point on each side of the intestinal wall contour curve.

7. The medical image based intelligent colon polyp margin delineation method of claim 1, wherein, After outlining the polyp contour on the inner wall of the colon in each frame of image, the method further includes: For the same polyp location point, determining the outline length of the colon inner wall polyp outlined in the current frame image and all previous frame images; Based on the contour lengths in all frame images before the current frame image, predicting the contour length in the current frame image to obtain an expected contour length; Based on the deviation between the contour length in the current frame image and the expected contour length, determining whether the contour of the colon inner wall polyp in the current frame image is complete; If the outline is incomplete, adjust the angle of the colonoscope to obtain a new internal colon detection image and re-outline the polyp on the inner wall of the colon.

8. The medical image based intelligent delineation of colonic polyp margin method of claim 7, wherein, Determine whether the outline of the colon polyp in the current frame image is complete, including: Determining a deviation threshold based on a standard deviation of the contour lengths in all frame images before the current frame image; If the deviation is greater than the deviation threshold, it is determined that the outline of the colon inner wall polyp in the current frame image is incomplete; otherwise, it is determined that the outline of the colon inner wall polyp in the current frame image is complete.

9. The medical image based intelligent colon polyp margin delineation method of claim 1, wherein, The method further comprises: Based on the outline of the colonic polyp, the colonic polyps were classified.

Images