Intelligent edge sketching method for colon polyps based on medical images

Abstract

The present application relates to the technical field of edge segmentation, and particularly relates to a colon polyp edge intelligent sketching method based on medical images, which comprises the following steps: in the process of colonoscopy, edge detection is performed on each frame of image in the collected colon internal detection image data to obtain a plurality of nested intestinal wall contour curves; based on the normal direction change of each position point on the intestinal wall contour curve, the normal deviation degree of each position point is determined, and then the polyp position point on the intestinal wall contour curve is determined; based on the curvature change and the normal direction change of the adjacent position points of the polyp position point on the intestinal wall contour curve, the polyp edge end point of the polyp position point is determined, and the colon inner wall polyp contour sketching is performed in each frame of image. The present application accurately locates the polyp protruding area on the intestinal wall and performs edge contour sketching, thereby effectively improving the polyp edge recognition accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of edge segmentation, and in particular to a colon polyp edge intelligent delineation method based on medical images. BACKGROUND

[0002] In the clinical diagnosis and treatment system, the screening and early diagnosis of colon polyps have a milestone significance for the prevention and treatment of colorectal cancer. Medical research has confirmed that about 80% to 95% of colorectal cancer is evolved from adenomatous polyps, and early detection and timely implementation of endoscopic resection can greatly reduce the risk of colon cancer.

[0003] The traditional colon polyp identification method mainly relies on the naked eye observation and experience judgment of endoscopists, and the polyps are identified artificially through the characteristics such as the shape, color and surface texture in the endoscopic images. However, this method has obvious limitations: on the one hand, visual assessment is easily affected by subjective experience, and the identification sensitivity for small or flat polyps is insufficient; on the other hand, when the polyp boundary is fuzzy or the contrast with the surrounding mucosa is low, the naked eye is difficult to accurately define the polyp edge, which may lead to missed diagnosis or misjudgment. SUMMARY

[0004] In order to solve the above technical problem of difficult to accurately define the polyp edge, the purpose of the present application is to provide a colon polyp edge intelligent delineation method based on medical images, and the technical solution adopted is as follows:

[0005] In the first aspect, the present application provides a colon polyp edge intelligent delineation method based on medical images, comprising the following steps:

[0006] During the colonoscopy process, edge detection is performed on each frame of image in the collected colon internal detection image data to obtain a plurality of nested intestinal wall contour curves;

[0007] Based on the normal direction change of each position point on the intestinal wall contour curve, the normal deviation degree of each position point on the intestinal wall contour curve is determined;

[0008] Based on the normal deviation degree, the polyp position point on the intestinal wall contour curve is determined;

[0009] Based on the curvature change and normal direction change of the adjacent position points of the polyp position point on the intestinal wall contour curve, the polyp edge end point of the polyp position point is determined;

[0010] Based on the polyp position point and its polyp edge end point, colon inner wall polyp contour delineation is performed in each frame of image;

[0011] Determining the normal deviation degree of each position point on the intestinal wall contour curve comprises:

[0012] A center point of the innermost intestinal wall contour curve is taken as a starting point to construct a plurality of radial lines;

[0013] An intersection point of each of the radial lines and the plurality of intestinal wall contour curves is determined;

[0014] A normal deviation degree of a position point on the intestinal wall contour curve where the intersection point is located is determined based on a normal direction difference between the intersection point and other position points on the intestinal wall contour curve.

[0015] In combination with the first aspect, in some possible implementation manners, the normal deviation degree of the position point on the intestinal wall contour curve where the intersection point is located is determined by:

[0016] Curve fitting is performed on the intestinal wall contour curve to obtain a fitting equation;

[0017] Normal position points are determined according to a condition that each of the other position points on the intestinal wall contour curve meets the fitting equation;

[0018] An average value and a standard deviation of angles corresponding to normal directions of all the normal position points on the intestinal wall contour curve are determined to obtain an angle average value and an angle standard deviation;

[0019] An angle difference value is determined by determining an absolute value of a difference between an angle corresponding to a normal direction of the intersection point on the intestinal wall contour curve and the angle average value;

[0020] The normal deviation degree of the position point on the intestinal wall contour curve where the intersection point is located is determined by determining a ratio of the angle difference value to the angle standard deviation.

[0021] In combination with the first aspect, in some possible implementation manners, the polyp edge end point of the polyp position point is determined by:

[0022] For an arbitrary position point on two sides of the polyp position point on the intestinal wall contour curve, a polyp edge end point score of the arbitrary position point is determined based on a curvature difference and a normal direction difference between the arbitrary position point and a neighboring position point on a side close to the polyp position point, and a curvature difference and a normal direction difference of each position point in a neighboring window region of the arbitrary position point;

[0023] A target neighboring position point closest to the polyp position point in each side neighboring position point of the polyp position point on the intestinal wall contour curve and having a polyp edge end point score greater than a score threshold value is determined as a polyp edge end point.

[0024] In combination with the first aspect, in some possible implementation manners, the polyp edge end point score of the arbitrary position point is determined by:

[0025] determining a curvature difference value between the arbitrary position point and a neighboring position point on a side of the arbitrary position point close to the polyp position point, and determining an angle difference value between a normal direction corresponding angle of the arbitrary position point and a normal direction corresponding angle of the neighboring position point;

[0026] determining an average curvature difference value of the curvature difference values of the position points in a neighboring window region of the arbitrary position point, to obtain an average curvature difference;

[0027] determining an average angle difference value of the angle difference values of the position points in the neighboring window region of the arbitrary position point, to obtain an average angle difference;

[0028] determining a first ratio value of the curvature difference value and the average curvature difference;

[0029] determining a second ratio value of the angle difference value and the average angle difference;

[0030] performing weighted addition on the first ratio value and the second ratio value by using a dynamic weight coefficient, to obtain a polyp edge end point score of the arbitrary position point.

[0031] In combination with the first aspect, in some possible implementation manners, the determination process of the dynamic weight coefficient comprises:

[0032] judging a relationship between the curvature of the arbitrary position point on the intestinal wall contour curve and the average curvature difference;

[0033] if it is judged that the curvature of the arbitrary position point is greater than a first set value multiple of the average curvature difference, setting the dynamic weight coefficient of the first ratio value to be greater than the dynamic weight coefficient of the second ratio value, and the first set value is greater than 1;

[0034] if it is judged that the curvature of the arbitrary position point is less than a second set value multiple of the average curvature difference, setting the dynamic weight coefficient of the first ratio value to be less than the dynamic weight coefficient of the second ratio value, and the second set value is less than 1;

[0035] otherwise, setting the dynamic weight coefficient of the first ratio value to be equal to the dynamic weight coefficient of the second ratio value.

[0036] In combination with the first aspect, in some possible implementation manners, the determination process of the score threshold value comprises:

[0037] determining a distance from each neighboring position point of the neighboring position points on each side of the intestinal wall contour curve to the polyp position point;

[0038] adjusting a set basic threshold value based on the distance, to obtain an adjusted threshold value;

[0039] determining a minimum value of a set upper threshold value and the adjusted threshold value;

[0040] the minimum value as each of the adjacent position points on each side of the intestinal wall profile curve.

[0041] In combination with the first aspect, in some possible implementation manners, after the polyp profile on the inner wall of the colon is outlined in each frame of image, the method further includes:

[0042] For the same polyp position point, the profile length of the polyp profile on the inner wall of the colon outlined in the current frame of image and all the frame images before the current frame of image is determined;

[0043] Based on the profile length in all the frame images before the current frame of image, the profile length in the current frame of image is predicted to obtain an expected profile length;

[0044] Based on the deviation amount of the profile length in the current frame of image and the expected profile length, it is determined whether the polyp profile on the inner wall of the colon in the current frame of image is outlined completely;

[0045] If the polyp profile on the inner wall of the colon is not outlined completely, the colonoscope angle is adjusted to obtain a new internal detection image of the colon to re-outline the polyp profile on the inner wall of the colon.

[0046] In combination with the first aspect, in some possible implementation manners, it is determined whether the polyp profile on the inner wall of the colon in the current frame of image is outlined completely, including:

[0047] Based on the standard deviation of the profile length in all the frame images before the current frame of image, a deviation threshold is determined;

[0048] If the deviation amount is greater than the deviation threshold, it is determined that the polyp profile on the inner wall of the colon in the current frame of image is not outlined completely, otherwise it is determined that the polyp profile on the inner wall of the colon in the current frame of image is outlined completely.

[0049] In combination with the first aspect, in some possible implementation manners, the method further includes:

[0050] Based on the outlined polyp profile on the inner wall of the colon, classification of the polyp on the inner wall of the colon is performed.

[0051] The second aspect, the present application also provides a kind of based on medical image's colon polyp edge intelligent outlining device, the device includes:

[0052] The first module is used to carry out edge detection on each frame of image in the collected internal detection image data of colon during colonoscopy, and obtains several nested intestinal wall profile curves;

[0053] The second module is used to determine the normal deviation degree of each position point on the intestinal wall profile curve based on the normal direction change of each position point on the intestinal wall profile curve.

[0054] a third module configured to determine a polyp position point on the intestinal wall contour curve based on the normal deviation degree;

[0055] a fourth module configured to determine a polyp edge end point of the polyp position point based on a curvature change and a normal direction change of adjacent position points of the polyp position point on the intestinal wall contour curve;

[0056] a fifth module configured to perform polyp contour delineation on the inner wall of the colon based on the polyp position point and the polyp edge end point thereof in each frame of image;

[0057] determining the normal deviation degree of each position point on the intestinal wall contour curve comprises:

[0058] constructing a plurality of radial lines with the center point of the innermost intestinal wall contour curve as a starting point;

[0059] determining the intersection point of each radial line and the plurality of intestinal wall contour curves;

[0060] determining the normal deviation degree of the position point where the intersection point is located on the intestinal wall contour curve based on the normal direction difference between the intersection point and other position points on the intestinal wall contour curve.

[0061] In a third aspect, the present application further provides a colon polyp edge intelligent delineation system based on medical images, comprising a memory and a processor. The memory is configured to store executable computer program code, and the processor is configured to call and run the executable computer program code from the memory, so that the system executes the method in the first aspect or any possible implementation manner of the first aspect.

[0062] In a fourth aspect, the present application further provides a computer program product, which comprises computer program code. When the computer program code is run on a computer, the computer executes the method in the first aspect or any possible implementation manner of the first aspect.

[0063] In a fifth aspect, the present application further provides a computer readable storage medium, which stores computer program code. When the computer program code is run on a computer, the computer executes the method in the first aspect or any possible implementation manner of the first aspect.

[0064] The present application has the following beneficial effects: in the process of colonoscopy, by acquiring the intestinal wall contour curve of each frame of image in the colon internal detection image data, and based on the normal direction change of each position point on the intestinal wall contour curve, the polyp position point on the intestinal wall contour curve is accurately identified; meanwhile, based on the curvature change and the normal direction change of the adjacent position points of the polyp position point on the intestinal wall contour curve, the polyp edge end point of the polyp position point is accurately identified; finally, based on the polyp position point and the polyp edge end point, the complete outlining of the colon inner wall polyp contour is carried out. The present application accurately locates the polyp position point and the polyp edge end point on the intestinal wall and carries out the complete outlining of the edge contour, thereby effectively improving the polyp edge recognition accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0065] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, and the advantages thereof, below, a brief introduction will be given to the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor based on these drawings.

[0066] Figure 1 A step flow chart of a medical image-based colon polyp edge intelligent outlining method according to an embodiment of the present application;

[0067] Figure 2 A schematic diagram of a plurality of intestinal wall contour curves constructed in a certain frame of image according to an embodiment of the present application;

[0068] Figure 3 A schematic diagram of the normal direction of the intestinal wall contour curve and the polyp edge at a certain polyp position point according to an embodiment of the present application;

[0069] Figure 4 A structural schematic diagram of a medical image-based colon polyp edge intelligent outlining system according to an embodiment of the present application. DETAILED DESCRIPTION

[0070] In order to clearly illustrate the technical features of the present application, below, the present application will be described in detail through specific embodiments and in combination with the drawings.

[0071] Below, the embodiments of the present application will be described in more detail with reference to the drawings. Although some embodiments of the present application are shown in the drawings, it should be understood that the present application can be realized in various forms, and should not be interpreted as being limited to the embodiments described herein, on the contrary, these embodiments are provided to make the present application more thorough and complete. It should be understood that the drawings and embodiments of the present application are only for exemplary purposes, and are not intended to limit the protection scope of the present application.

[0072] It should be understood that each of the steps recited in the method embodiments of the present application can be performed in different orders and / or in parallel. In addition, the method embodiments can include additional steps and / or omit performing the steps shown. The scope of the present application is not limited in this respect.

[0073] The term "comprising" and variations thereof as used herein are open-ended, that is "including but not limited to". The term "based on" is "based, at least in part, on". The term "one embodiment" means "at least one embodiment". The term "another embodiment" means "at least one additional embodiment". The term "some embodiments" means "at least some embodiments". Related terms have analogous meanings.

[0074] It should be noted that the terms "first", "second", and so on used in the present application are only used to distinguish different devices, modules or units, and are not intended to limit the order or interdependence of the functions performed by these devices, modules or units.

[0075] Although the operations or steps in the embodiments of the present application are described in a particular order in the accompanying drawings, it should not be understood as requiring the operations or steps to be performed in the particular order or serial order shown, or requiring all of the operations or steps to be performed to obtain a desired result. In the embodiments of the present application, the operations or steps can be performed in series; the operations or steps can be performed in parallel; or a part of the operations or steps can be performed.

[0076] At the same time, it should be understood that the data involved in the technical solutions of the present application (including but not limited to the data itself, the acquisition or use of the data) should comply with the requirements of the corresponding laws, regulations and relevant provisions. Unless otherwise defined, all technical and scientific terms used in the present application have the same meaning as understood by those skilled in the art to which the present application belongs, and all parameters or indicators in the formulas involved in the present application are normalized values that eliminate the dimension influence.

[0077] In order to solve the problem of difficult accurate definition of the edge of the polyp, the present application provides a medical image-based intelligent outlining method for the edge of the colon polyp. The method analyzes the colon internal detection image data shot by a colonoscope, identifies a plurality of intestinal wall profile curves embedded in each frame of image, accurately locates the polyp position point and the polyp edge end on the intestinal wall, and performs complete contour outlining of the edge, thereby effectively improving the polyp edge recognition accuracy and providing a reliable basis for the doctor to judge the colon condition of the patient.

[0078] In the following, a medical image-based intelligent outlining method for the edge of the colon polyp provided by the embodiments of the present application will be described in detail with reference to the accompanying drawings.

[0079] Figure 1A basic flowchart of a medical image-based colon polyp edge intelligent sketching method provided by an embodiment of the present application is shown, as shown in Figure 1 The method specifically includes the following steps:

[0080] Step S100: During the colonoscopy process, edge detection is performed on each frame of image in the collected colon internal detection image data, to obtain a plurality of nested intestinal wall contour curves.

[0081] Specifically, in order to realize accurate sketching of the colon polyp edge, a standardized process is adopted to obtain high-quality colon internal detection image data of the patient, that is, the patient is adjusted to a low residue diet 3 days before the examination, and the colon preparation is completed 1 day before the examination. During the colonoscopy process, the high-definition camera of the colonoscope is used to dynamically collect the colon internal intestinal mucosa images at a speed of 30 frames per second, and image enhancement techniques such as narrowband imaging are used to enhance the collected images to improve the image contrast. The enhanced detection images are stored in real time in the medical image system, providing continuous frame colon internal detection image data for subsequent polyp edge sketching.

[0082] During the colonoscopy process, edge detection is performed on each frame of image in the obtained colon internal detection image data, such as using the Canny edge detection algorithm for edge detection, and a plurality of continuous and nested intestinal wall contour curves in each frame of image are constructed. Figure 2 A plurality of intestinal wall contour curves constructed in a frame of image are shown. Subsequently, by analyzing the change characteristics of the plurality of intestinal wall contour curves constructed in each frame of image, the intestinal wall polyp region is accurately positioned and the edge is sketched.

[0083] Step S200: Based on the normal direction change of each position point on the intestinal wall contour curve, the normal deviation degree of each position point on the intestinal wall contour curve is determined.

[0084] Specifically, under normal circumstances, the inner wall surface of the healthy colon is smooth, and its structure is shown as a series of nested circular contours on a two-dimensional image. These contour edges are presented in the form of one nest after another, reflecting the hierarchical structure of the intestinal wall. However, in the colonoscopy image where polyps may occur, the regular contour may be locally deformed, causing the normal direction of the local contour to change. Therefore, for each frame of image in the colon internal detection image data, the normal direction change of each position point on the intestinal wall contour curve in the image is analyzed, and the normal deviation degree is determined to quantify the degree of deviation of the normal direction of the position point from the normal condition, so as to accurately identify the position point where the polyp appears.

[0085] Further, in a possible implementation, determining the normal deviation degree of each position point on the intestinal wall contour curve includes:

[0086] Step S201: Constructing several radial lines with the center point of the innermost intestinal wall contour curve as the starting point.

[0087] Specifically, in order to more accurately analyze the normal line deviation of each position point on the intestinal wall contour curve, several radial lines are constructed with the center point of the innermost intestinal wall contour curve as the starting point. In a specific example, a polar coordinate system is established with the center point of the innermost intestinal wall contour curve as the origin and the horizontal right as the polar axis. Starting from the polar axis, rotate in the clockwise direction, and draw a radial line along the polar axis every time a set angle is rotated, thereby constructing several radial lines. The size of the set angle can be reasonably set according to the need to balance the detection accuracy and the amount of calculation.

[0088] In a specific example, the set angle is determined by the following formula:

[0089] ;

[0090] In the formula: represents the basic angle interval (by default ); represents the average radius of curvature of all intestinal wall contour curves in each frame of image; represents the average radius of curvature of the normal intestinal wall.

[0091] In the above formula, when the local radius of curvature of the intestinal wall in the image is small, the angle is reduced by the formula, thereby increasing the sampling points in the image to capture more subtle contour deformations; on the contrary, when the radius of curvature of the intestinal wall is close to the normal value, a larger angle is maintained to reduce calculation redundancy.

[0092] Step S202: Determine the intersection points of each radial line and several intestinal wall contour curves.

[0093] Specifically, the radial lines will pass through each intestinal wall contour curve in turn and produce intersection points. Assuming that the inner wall of the colon in a frame of image is represented by I nested intestinal wall contour curves, the intersection set of the nth radial line and the I nested intestinal wall contour curves is , where represents the intersection point of the nth radial line and the i-th intestinal wall contour curve.

[0094] Step S203: Based on the difference in the normal line direction between the intersection points and other position points on the intestinal wall contour curve, determine the normal line deviation degree of the position point where the intersection point is located on the intestinal wall contour curve.

[0095] Specifically, since the normal direction of the normal intestinal wall profile is concentrated in the statistical reference range of the ellipse (determined by the mean and standard deviation), the normal direction of the polyp region deviates from the reference range due to the sharp change in local curvature. Therefore, by analyzing the difference in the normal direction between the intersection points and other position points on the intestinal wall profile curve, the normal deviation degree of the position points where the intersection points are located is determined to evaluate the deviation degree of the normal direction of the position points where the intersection points are located from the normal case.

[0096] In a specific example, determining the normal deviation degree of the position points where the intersection points on the intestinal wall profile curve are located includes: performing curve fitting on the intestinal wall profile curve to obtain a fitting equation; determining normal position points according to the fitting equation satisfied by each position point on the intestinal wall profile curve; determining the mean and standard deviation of the angles corresponding to the normal directions of all normal position points on the intestinal wall profile curve to obtain the angle mean and angle standard deviation; determining the absolute value of the difference between the angle corresponding to the normal direction of the intersection point on the intestinal wall profile curve and the angle mean to obtain an angle difference value; and determining the ratio of the angle difference value to the angle standard deviation to obtain the normal deviation degree of the position points where the intersection points on the intestinal wall profile curve are located.

[0097] In this specific example, since the polyp is a local abnormal protruding lesion, it only occupies a very small part of the intestinal wall region, so the normal region can be segmented by global geometric features, that is, an coordinate system is constructed, and an ellipse fitting equation is obtained by fitting each intestinal wall profile curve: wherein, represents the center coordinates of the fitted ellipse, and are the semi-axis lengths of the fitted ellipse on the x-axis and y-axis, respectively. The coordinates of each position point on each intestinal wall profile curve are substituted into the fitting equation, and if wherein, represents the tolerance for controlling inward concave, such as setting , represents the tolerance for controlling outward convex, such as setting , the corresponding position point is determined as a normal position point, and thus each normal position point in each intestinal wall profile curve can be determined.

[0098] For any intersection point in the intersection point set , the normal deviation degree of the position point where each intersection point on the intestinal wall profile curve is located is determined by the following calculation formula:

[0099] ;

[0100] wherein, represents the intersection point the normal direction of the location point; the intersection point on the intestinal wall contour curve the angle corresponding to the normal direction of the location point; respectively represent the average angle and the standard deviation of the angle, i.e., the average value and the standard deviation of the angle corresponding to the normal direction of all normal location points on the intestinal wall contour curve.

[0101] Step S300: determining the polyp location point on the intestinal wall contour curve based on the normal deviation degree.

[0102] Specifically, the greater the value of the normal deviation degree, the more likely the corresponding intersection point is an abnormal point in the polyp region. Therefore, based on the normal deviation degree, the polyp location point on the intestinal wall contour curve can be identified. In a specific example, a preset deviation degree threshold T is set, such as T=2 or 3. The normal deviation degree of the location point of each intersection point in the intersection point set on the intestinal wall contour curve is compared with the preset deviation degree threshold T. When the normal deviation degree is greater than the preset deviation degree threshold T, the location point of the corresponding intersection point on the intestinal wall contour curve is determined as the polyp location point. In this way, by traversing any intersection point in the intersection point set corresponding to each radiation line, abnormal points exceeding the statistical benchmark range are screened out. These abnormal points are the location points in the polyp region, and the edge of the colon polyp region can be outlined based on these points.

[0103] Step S400: determining the polyp edge end point of the polyp location point based on the curvature change and the normal direction change of the adjacent location point of the polyp location point on the intestinal wall contour curve.

[0104] Specifically, after the intersection point in the polyp region is located, the pixel points are traversed in the left and right directions along the intestinal wall contour curve from the starting point, and the normal direction and the curvature value are recorded one by one. When the traversal point is on the polyp edge, the normal direction points to the inside of the polyp, and the curvature value may be higher or lower than the curvature of the circular inner wall according to the diversity of the polyp shape; when the traversal point reaches the circular inner wall, the normal direction points to the center of the colon inner wall, and the curvature value is constant. Figure 3 A normal direction diagram of the intestinal wall contour curve and the polyp edge where a certain polyp location point is located is shown.

[0105] Therefore, the polyp edge end point can be extracted by analyzing the difference between the two types of features to construct the polyp contour. Since a single feature (such as only curvature or normal direction) cannot cover all polyp shapes in the case of unknown polyp shape, the applicability can be improved by complementary features, and the joint determination of curvature and normal direction is realized, so as to accurately identify the polyp edge end point of the polyp location point on the intestinal wall contour curve.

[0106] Further, in a possible implementation, the polyp edge end point of the polyp position point is determined, including:

[0107] Step S401: For any one position point on the two sides of the polyp position point on the intestinal wall contour curve, based on the curvature difference and the normal direction difference between the any one position point and the adjacent position point on the side close to the polyp position point, and the curvature difference and the normal direction difference of each position point in the adjacent window region of the any one position point, the polyp edge end point score of the any one position point is determined.

[0108] Specifically, taking the polyp position point on the intestinal wall contour curve as the starting point, the pixel points are traversed in the left and right directions along the intestinal wall contour curve. In the traversal process, when the curvature and the normal direction of a certain position point change greatly relative to the curvature and the normal direction of the previous position point, it means that the position point is most likely to be the polyp edge end point. Therefore, by comparing the curvature difference and the normal direction difference between each position point and the previous position point with the curvature difference and the normal direction difference of the surrounding position points, the polyp edge end point score of each position point can be obtained, and the polyp edge end point can be accurately identified based on the polyp edge end point score.

[0109] In a specific example, the polyp edge end point score of the any one position point is determined, including: determining the curvature difference value and the angle difference value of the corresponding angle of the normal direction between the any one position point and the adjacent position point on the side close to the polyp position point; determining the average value of the curvature difference of each position point in the adjacent window region of the any one position point to obtain the average curvature difference; determining the average value of the angle difference value of each position point in the adjacent window region of the any one position point to obtain the average angle difference; determining the ratio of the curvature difference value to the average curvature difference to obtain a first ratio; determining the ratio of the angle difference value to the average angle difference to obtain a second ratio; and weighting and adding the first ratio and the second ratio by using a dynamic weight coefficient to obtain the polyp edge end point score of the any one position point.

[0110] In the specific example, the polyp edge end point score of the any one position point on the two sides of the polyp position point on the intestinal wall contour curve is calculated by the following calculation formula:

[0111] ;

[0112] In the formula: represents the polyp edge end point score of the any one position point on the two sides of the polyp position point on the intestinal wall contour curve; represents the curvature difference value between the any one position point and the adjacent position point on the side close to the polyp position point, i.e., the absolute value of the curvature difference between the any one position point and the previous adjacent position point in the traversal process; the angle difference value between the normal direction corresponding angle of any one position point on the two sides and the adjacent position point on the side close to the polyp position point, i.e. the absolute value of the angle difference between the normal direction corresponding angle of any one position point on the two sides and the previous adjacent position point in the traversal process; the average value of the curvature difference of each position point in the adjacent window region (such as a 5-pixel window centered on any one position point) of any one position point on the two sides, i.e. the average curvature difference of each position point in the adjacent window region of any one position point on the two sides; the average value of the angle difference value of each position point in the adjacent window region (such as a 5-pixel window centered on any one position point) of any one position point on the two sides, i.e. the average angle difference of each position point in the adjacent window region of any one position point on the two sides; 、 the dynamic weight coefficient, and α+β=1.

[0113] In the above formula, the dynamic weight coefficient 、 adjusts the influence proportion of the two factors of curvature and normal direction to accurately determine the polyp edge end point score of any one position point on the two sides of the polyp position point on the intestinal wall contour curve. The specific value of the dynamic weight coefficient 、 is determined according to the relationship between the curvature of any one position point on the two sides of the polyp position point on the intestinal wall contour curve and the average curvature difference . Since the junction end point of the polyp and the normal intestinal wall, i.e. the polyp edge end point, is the key position where the polyp shape transitions from abnormal to normal, the morphological mutation characteristics of the edge end point are significant, resulting in the simultaneous increase of the curvature change parameters and in the above formula, which shows that the score is higher.

[0114] Further, in a possible implementation, the determination process of the dynamic weight coefficient 、 includes: judging the relationship between the curvature of any one position point on the intestinal wall contour curve and the average curvature difference; if it is judged that the curvature of any one position point is greater than the first set value multiple of the average curvature difference, setting the dynamic weight coefficient of the first ratio to be greater than the dynamic weight coefficient of the second ratio, and the first set value is greater than 1; if it is judged that the curvature of any one position point is less than the second set value multiple of the average curvature difference, setting the dynamic weight coefficient of the first ratio to be less than the dynamic weight coefficient of the second ratio, and the second set value is less than 1; otherwise, setting the dynamic weight coefficient of the first ratio to be equal to the dynamic weight coefficient of the second ratio.

[0115] In a specific example, the first set value is set to 2 and the second set value is set to 0.5, when the curvature of any one position point is greater than 2 times the average curvature difference, the dynamic weight coefficient of the first ratio is set to be greater than the dynamic weight coefficient of the second ratio, and when the curvature of any one position point is less than 0.5 times the average curvature difference, the dynamic weight coefficient of the first ratio is set to be less than the dynamic weight coefficient of the second ratio. a dynamic weight coefficient is set when the curvature of any one position point is , a dynamic weight coefficient is set when the curvature of any one position point is , , a dynamic weight coefficient is set when the curvature of any one position point is , .

[0116] Step S402: determining the target adjacent position point closest to the polyp position point, in which the polyp edge end point score of each side adjacent position point of the polyp position point on the intestinal wall contour curve is greater than the score threshold, as the polyp edge end point.

[0117] Specifically, in the process of traversing the surrounding pixel points along the intestinal wall contour curve in the left and right directions of the polyp position point, the polyp edge end point score of each position point on both sides of the polyp position point can be obtained in turn, and when the polyp edge end point score is greater than the score threshold, the corresponding position point is taken as the target adjacent position point, which is the polyp edge end point of the polyp position point.

[0118] In the process of traversing the surrounding pixel points along the intestinal wall contour curve in the left and right directions of the polyp position point, as the traversal distance increases, the traversed pixel points gradually move away from the starting point, and the long-distance traversal is easily disturbed by noise, which may lead to misjudgment of the noise as the edge end point. Therefore, as the traversal distance increases, the score threshold is dynamically improved to avoid misjudgment of the pixel points as the polyp edge end point in these long-distance regions.

[0119] Further, in a possible implementation, the determination process of the score threshold comprises: determining the distance from each adjacent position point in the adjacent position points on both sides of the intestinal wall contour curve to the polyp position point; adjusting the set basic threshold based on the distance to obtain an adjusted threshold; determining the minimum value of the set upper limit threshold and the adjusted threshold; and taking the minimum value as the score threshold of each adjacent position point in the adjacent position points on both sides of the intestinal wall contour curve.

[0120] In a specific example, the score threshold is calculated by the following formula:

[0121] ;

[0122] In the formula: The set upper limit threshold is set to prevent extreme noise or characteristic fluctuations from causing the score threshold to increase indefinitely, ensuring stability, and is usually set to 1.5; The set basic threshold is set to be relatively low, such as 0.8, to reduce the risk of missed detection; and The set basic threshold is set to be relatively low, such as 0.8, to reduce the risk of missed detection. denotes the distance attenuation coefficient, which needs to be calibrated in combination with actual data, and is usually taken as 0.1-0.4; denotes the distance from each of the adjacent position points on each side of the polyp position point on the intestinal wall contour curve to the polyp position point, i.e., the number of pixel points from each of the adjacent position points on each side of the polyp position point on the intestinal wall contour curve to the polyp position point.

[0123] In the above manner, by determining the polyp edge end point score of any position point on the two sides of the polyp position point on the intestinal wall contour curve and comparing the polyp edge end point score with the score threshold value determined adaptively, the polyp edge end points on the two sides of the polyp position point on the intestinal wall contour curve can be accurately identified.

[0124] Step S500: Based on the polyp position point and the polyp edge end point thereof, the polyp contour on the inner wall of the colon is outlined in each frame of image.

[0125] Specifically, starting from the polyp position point on the intestinal wall contour curve, the surrounding pixel points are tracked in the left and right directions along the intestinal wall contour curve until the polyp edge end point is reached, thereby realizing the contour outlining of the polyp region on the inner wall of the colon.

[0126] When the colonoscope is pushed forward in the colon, if the polyp contour on the inner wall of the colon is complete, the contour should show a gradual enlargement in the continuous multiple frames of images. However, when the polyp is partially blocked by the intestinal wall folds and other structures, the contour will show a sudden increase during the movement of the colonoscope. Based on this dynamic change characteristic, the integrity of the outlined polyp contour on the inner wall of the colon needs to be verified.

[0127] Further, after the polyp contour on the inner wall of the colon is outlined in each frame of image, the method further comprises:

[0128] Step S501: For the same polyp position point, the contour length of the polyp contour on the inner wall of the colon outlined in the current frame of image and all the previous frames of image is determined.

[0129] Specifically, the contour length of the polyp contour on the inner wall of the colon outlined in each frame of image after the polyp appears is counted.

[0130] Step S502: Based on the contour length in all the previous frames of image, the contour length in the current frame of image is predicted to obtain an expected contour length.

[0131] Specifically, for the same polyp position point, based on the change of the contour length in all the previous frames of image, the contour length change law under normal circumstances is established as a reference benchmark for subsequent detection.

[0132] In a specific example, for the same polyp location, the contour length growth rate is calculated based on the contour lengths in all previous frames using the following formula:

[0133] ;

[0134] In the formula: L(N) represents the contour length growth rate; L(N) represents the contour length in the Nth frame before the current frame; L(1) represents the contour length in the 1st frame before the current frame; N represents the number of frames before the current frame after the polyp location point is detected.

[0135] Furthermore, for the same polyp location, the contour length in the previous frame image is used as the basis. and contour length growth rate Determine the expected contour length in the current frame image. .

[0136] Step S503: Based on the deviation between the contour length in the current frame image and the expected contour length, determine whether the contour of the colon wall polyp in the current frame image is completely drawn.

[0137] Specifically, the deviation is calculated by dividing the absolute value of the difference between the outline length in the current frame image and the expected outline length by the expected outline length. If the deviation between the outline length of the polyp on the colon wall in the current frame image and the expected outline length is small, it indicates that the outline of the polyp on the colon wall in the current frame image is complete; otherwise, it indicates that the outline is complete.

[0138] Furthermore, in one possible implementation, determining whether the outline of the colonic wall polyp in the current frame image is completely drawn includes: determining a deviation threshold based on the standard deviation of the outline length in all previous frames; if the deviation is greater than the deviation threshold, then the outline of the colonic wall polyp in the current frame image is determined to be incomplete; otherwise, the outline of the colonic wall polyp in the current frame image is determined to be complete. In a specific example, the standard deviation of the outline length in all previous frames is determined. Obtain the deviation threshold When the deviation is greater than the deviation threshold, If the outline length changes abruptly, it is considered an abnormal drawing; otherwise, it is considered a normal drawing.

[0139] Step S504: If the outline is incomplete, adjust the colonoscope angle to obtain a new image of the inside of the colon and redraw the outline of the polyp on the colon wall.

[0140] Specifically, if the polyp contouring is incomplete, the colonoscope angle needs to be adjusted to reacquire the colon interior detection image to re-perform the colon inner wall polyp contouring.

[0141] After the colon inner wall polyp contouring is completed, the colon inner wall polyp is classified based on the contoured colon inner wall polyp contour. In a specific example, the classification process includes: according to the ratio R / L of the curvature radius R of the polyp contour under the colonoscope to the base length L (i.e. the distance between the two points where the polyp contour connects with the intestinal wall contour), the polyp can be divided into three categories: the protruding type (R / L≥0.5), the base is wide and the surface is hemispherical protrusion, which is often seen in adenomas with a diameter >1cm; the sub-stem type (0.3≤R / L<0.5), the neck forms obvious narrowing but is not completely free, often accompanied by surface villous structure; the long-stem type (R / L<0.3), the length of the stem is more than twice the diameter of the lesion, the blood vessel runs in a spiral shape, and is easy to twist and bleed.

[0142] In addition, according to the number combination of each type of polyp in a single examination, risk stratification can be performed: low-risk group (1-2): if they are all <5mm protruding tubular adenomas, the 5-year cancer risk is <1%, and 3-year follow-up is recommended; medium-risk group (3-4): if it contains ≥1 sub-stem type villous adenoma, it needs to be combined with the Ki-67 proliferation index, and the positive one needs to be rechecked within 1 year; high-risk group (≥5 or any one >2cm): it suggests the possibility of hereditary polyposis, and APC gene detection and total colectomy planning are recommended.

[0143] Based on the same inventive concept, the embodiments of the present application also provide a medical image-based colon polyp edge intelligent contouring device, which comprises:

[0144] The first module is configured to perform edge detection on each frame of image in the collected colon interior detection image data during the colonoscopy process, to obtain a plurality of nested intestinal wall contour curves;

[0145] The second module is configured to determine the normal line deviation degree of each position point on the intestinal wall contour curve based on the normal line direction change of each position point on the intestinal wall contour curve;

[0146] The third module is configured to determine the polyp position point on the intestinal wall contour curve based on the normal line deviation degree;

[0147] The fourth module is configured to determine the polyp edge end point of the polyp position point based on the curvature change and the normal line direction change of the adjacent position points of the polyp position point on the intestinal wall contour curve;

[0148] The fifth module is configured to perform colon inner wall polyp contouring in each frame of image based on the polyp position point and the polyp edge end point thereof;

[0149] determining the normal line deviation degree of each position point on the intestinal wall profile curve, comprising:

[0150] taking the center point of the innermost intestinal wall profile curve as a starting point, constructing a plurality of radial lines;

[0151] determining the intersection of each radial line and the plurality of intestinal wall profile curves;

[0152] based on the difference in the direction of the normal line between the intersection and other position points on the intestinal wall profile curve, determining the normal line deviation degree of the position point where the intersection is located on the intestinal wall profile curve.

[0153] It should be noted that: the device provided in the above embodiment is only exemplified by the division of the above functional modules, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the computer device is divided into different functional modules to complete all or part of the functions described above.

[0154] Based on the same inventive concept, the embodiments of the present application also provide a colon polyp edge intelligent sketching system based on medical images, as shown in the following formula (I), the system comprises: a memory 401, a processor 402 and a computer program code 403 stored in the memory 401 and running on the processor 402, wherein when the processor 402 executes the computer program code 403, the system can execute any one of the above-mentioned colon polyp edge intelligent sketching methods based on medical images. Figure 4

[0155] The embodiments of the present application can divide the system into functional modules according to the above method examples, for example, each functional module can be corresponding, or two or more functions can be integrated in one processing module, and the integrated module can be realized in the form of hardware. It should be noted that the division of modules in this embodiment is illustrative, and is only a logical function division, and another division mode can be used in actual implementation.

[0156] Based on the same inventive concept, the embodiments of the present application also provide a computer program product, which comprises: computer program code, when the computer program code runs on a computer, the computer executes any one of the above-mentioned colon polyp edge intelligent sketching methods based on medical images.

[0157] Based on the same inventive concept, the embodiments of the present application also provide a computer readable storage medium, which stores computer program code, when the computer program code runs on a computer, the computer executes any one of the above-mentioned colon polyp edge intelligent sketching methods based on medical images. ​

[0158] It should be noted that the above-described embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalent ones; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method for intelligent delineation of colon polyp edges based on medical images, characterized in that, Includes the following steps: During a colonoscopy, edge detection is performed on each frame of the acquired images of the inside of the colon to obtain several nested intestinal wall contour curves. Based on the change in the normal direction at each point on the intestinal wall contour curve, the degree of deviation of the normal at each point on the intestinal wall contour curve is determined. Based on the degree of deviation of the normal, the location points of polyps on the intestinal wall contour curve are determined; Based on the curvature changes and normal direction changes of adjacent points on the intestinal wall contour curve, the endpoint of the polyp edge at the polyp location is determined. Based on the location points of the polyps and the endpoints of their edges, the outlines of the polyps on the inner wall of the colon are drawn in each frame of the image. Determining the degree of deviation of the normal at various points on the intestinal wall contour curve includes: Several radial lines are constructed, starting from the center point of the innermost intestinal wall contour curve; Determine the intersection points of each of the aforementioned radial lines and the plurality of intestinal wall contour curves; Based on the difference in normal direction between the intersection point and other points on the intestinal wall contour curve, the degree of normal deviation of the point where the intersection point is located on the intestinal wall contour curve is determined.

2. The intelligent delineation method for colon polyp edges based on medical images according to claim 1, characterized in that, Determining the degree of deviation of the normal at the intersection point on the intestinal wall contour curve includes: Curve fitting was performed on the intestinal wall contour curve to obtain the fitting equation; Based on the conformity of other points on the intestinal wall contour curve with the fitting equation, the normal location points are determined; Determine the average and standard deviation of the angles corresponding to the normal directions of all normal locations on the intestinal wall contour curve to obtain the mean and standard deviation of the angles; The absolute value of the difference between the angle corresponding to the normal direction of the intersection point on the intestinal wall contour curve and the mean angle is determined to obtain the angle difference value. The ratio of the angle difference value to the angle standard deviation is determined to obtain the degree of deviation of the normal line at the intersection point on the intestinal wall contour curve.

3. The intelligent delineation method for colon polyp edges based on medical images according to claim 1, characterized in that, Determining the endpoint of the polyp edge at the polyp location point includes: For any point on either side of the polyp location on the intestinal wall contour curve, the polyp edge endpoint score of the arbitrary location is determined based on the curvature difference and normal direction difference between the arbitrary location and its adjacent location on the side closer to the polyp location, as well as the curvature difference and normal direction difference of each location within the window area adjacent to the arbitrary location. The nearest target neighboring point to the polyp location point on each side of the polyp location point on the intestinal wall contour curve with a polyp edge endpoint score greater than a score threshold is determined as the polyp edge endpoint.

4. The intelligent delineation method for colon polyp edges based on medical images according to claim 3, characterized in that, Determining the polyp edge endpoint score at any given location includes: Determine the curvature difference value and the angle difference value of the corresponding angle of the normal direction between any given location point and its adjacent location point on the side closer to the polyp location point; The average curvature difference is obtained by determining the average curvature difference among all points within the adjacent window region of any given location point. The average angle difference is obtained by determining the average angle difference of all points within the adjacent window area of ​​any given location point. Determine the ratio of the curvature difference value to the average curvature difference to obtain a first ratio; Determine the ratio of the angle difference value to the average angle difference to obtain a second ratio; The first ratio and the second ratio are weighted and summed using dynamic weighting coefficients to obtain the polyp edge endpoint score at any given location.

5. The intelligent delineation method for colon polyp edges based on medical images according to claim 4, characterized in that, The process of determining the dynamic weighting coefficients includes: Determine the relationship between the curvature at any point on the intestinal wall contour curve and the difference in average curvature; If it is determined that the curvature of any point is greater than a first set value multiple of the average curvature difference, the dynamic weighting coefficient of the first ratio is set to be greater than the dynamic weighting coefficient of the second ratio, and the first set value is greater than 1. If it is determined that the curvature of any point is less than a second set value multiple of the average curvature difference, the dynamic weighting coefficient of the first ratio is set to be less than the dynamic weighting coefficient of the second ratio, and the second set value is less than 1. Otherwise, the dynamic weighting coefficient of the first ratio is set to be equal to the dynamic weighting coefficient of the second ratio.

6. The intelligent delineation method for colon polyp edges based on medical images according to claim 3, characterized in that, The process of determining the score threshold includes: Determine the distance from each neighboring point on each side of the intestinal wall contour curve to the polyp location point; The adjusted threshold is obtained by adjusting the set base threshold based on the distance. Determine the minimum value between the set upper limit threshold and the adjustment threshold; The minimum value is taken as each of the nearest neighbor points on each side of the intestinal wall contour curve.

7. The intelligent delineation method for colon polyp edges based on medical images according to claim 1, characterized in that, After outlining the polyps on the colonic wall in each frame of the image, the method further includes: For the same polyp location point, determine the outline length of the colon wall polyp outline drawn in the current frame image and all previous frame images; Based on the contour lengths in all previous frames, the contour length in the current frame is predicted to obtain the expected contour length. Based on the deviation between the outline length in the current frame image and the expected outline length, it is determined whether the outline of the colon wall polyp in the current frame image is completely drawn. If the outline is incomplete, adjust the colonoscope angle to obtain new images of the inside of the colon and redraw the outline of the polyps on the colon wall.

8. The intelligent delineation method for colon polyp edges based on medical images according to claim 7, characterized in that, Determine whether the outline of the colon wall polyp in the current frame image is completely drawn, including: The deviation threshold is determined based on the standard deviation of the contour length in all previous frames of the current frame. If the deviation is greater than the deviation threshold, it is determined that the outline of the polyp on the colon wall in the current frame image is incomplete; otherwise, it is determined that the outline of the polyp on the colon wall in the current frame image is complete.

9. The intelligent delineation method for colon polyp edges based on medical images according to claim 1, characterized in that, The method further includes: Based on the outlined polyps of the colonic wall, polyps are classified.

Images