Method and device for colorectum curved surface registration and storage medium

Abstract

The invention discloses a method and equipment for colorectum curved surface registration and a storage medium. The method comprises the following steps: collecting an upward three-dimensional curved surface and a downward three-dimensional curved surface of the colorectum, and parameterizing the upward three-dimensional curved surface and the downward three-dimensional curved surface to corresponding plane domains respectively; extracting respective intestinal band feature points of the pitching three-dimensional curved surface and the pitching three-dimensional curved surface in the corresponding plane domains; based on the respective intestinal band feature points, determining a first intestinal band line and a second intestinal band line corresponding to the pitching three-dimensional curved surface and the pitching three-dimensional curved surface; and registering the first intestinal girdle line and the second intestinal girdle line to realize colorectum curved surface registration. By means of the scheme, an accurate and stable registration mode can be provided, and registration is conducted on the pitching position three-dimensional curved surface and the pitching position three-dimensional curved surface of the colorectum.

Description

Technical Field

[0001] The present application generally relates to the field of image processing technology. More specifically, the present application relates to a method, device and computer storage medium for colorectal curved surface registration. Background Art

[0002] Colorectal cancer is currently the second largest killer related to cancer. The precursor of colorectal cancer is often the appearance of polyps. Early detection, intervention and removal of polyps can greatly reduce the mortality rate of lesions. Traditional optical colonoscopy can observe and remove polyps, so it has been widely used. However, optical colonoscopy requires general anesthesia and is invasive and destructive. In addition, when patients undergo optical colonoscopy, many of them have already passed the optimal intervention period, so optical colonoscopy is not suitable for large-scale screening.

[0003] The virtual colonoscopy detection program based on Computer Aided Diagnosis (CAD) is currently the recommended method for large-scale screening. It is usually scanned using CT equipment and is therefore non-invasive. By analyzing and processing the three-dimensional colon surfaces acquired by the CT equipment, colon polyps can be effectively found and early warnings can be given. For virtual colonoscopy systems, surface registration can be conveniently used to compare unclear areas or confirm the location of polyps. However, when scanning with a CT device, although the inner wall of the intestine has been cleaned, there are still cleaning residues. It is usually necessary to scan the colon surfaces at multiple angles (such as prone posture, supine posture, etc.) through the CT device for registration to confirm the location of the polyps. Therefore, how to align the colon surfaces at multiple angles has become a technical problem that needs to be solved urgently.

[0004] In view of this, there is an urgent need to provide a solution for colorectal surface registration. By parameterizing the supine three-dimensional surface and the prone three-dimensional surface to the corresponding two-dimensional plane domain, the computational complexity of the three-dimensional space is simplified. Then, the colorectal surface is registered in the two-dimensional plane domain through important and easily identifiable structural intestinal lines, so as to provide an accurate and stable registration method to align the supine three-dimensional surface and the prone three-dimensional surface of the colorectum. Summary of the invention

[0005] In order to at least solve one or more of the technical problems mentioned above, the present application proposes a solution for colorectal curved surface registration in multiple aspects.

[0006] In a first aspect, the present application provides a method for colorectal surface registration, comprising: acquiring a supine three-dimensional surface and a prone three-dimensional surface of the colorectum, and parameterizing the supine three-dimensional surface and the prone three-dimensional surface to corresponding plane domains, respectively; extracting the intestinal feature points of the supine three-dimensional surface and the prone three-dimensional surface in the corresponding plane domain; determining the first intestinal line and the second intestinal line corresponding to the supine three-dimensional surface and the prone three-dimensional surface based on the respective intestinal feature points; and registering the first intestinal line and the second intestinal line to achieve colorectal surface registration.

[0007] In one embodiment, extracting the intestinal feature points of the supine three-dimensional surface and the prone three-dimensional surface in the corresponding plane domain includes: constructing a height map corresponding to the supine three-dimensional surface and the prone three-dimensional surface in the corresponding plane domain, wherein the height map represents the distance of the intestinal wall point on the supine three-dimensional surface or the prone three-dimensional surface from the central axis of the surface; extracting respective feature rows from the height maps corresponding to the supine three-dimensional surface and the prone three-dimensional surface; and extracting respective intestinal feature points of the supine three-dimensional surface and the prone three-dimensional surface based on the respective feature rows.

[0008] In another embodiment, extracting respective feature rows from the height maps corresponding to the supine three-dimensional curved surface and the prone three-dimensional curved surface respectively comprises: accumulating pixel values ​​of each row in the height maps corresponding to the supine three-dimensional curved surface and the prone three-dimensional curved surface respectively; calculating the local maximum value in the accumulated results in the height maps corresponding to the supine three-dimensional curved surface and the prone three-dimensional curved surface respectively; and extracting respective feature rows of the supine three-dimensional curved surface and the prone three-dimensional curved surface based on the points of the local maximum value.

[0009] In another embodiment, extracting the intestinal feature points of the supine three-dimensional surface and the prone three-dimensional surface based on the feature row includes: extracting a target point whose local minimum value is less than a preset threshold from the feature row; and determining the point among the target points whose rotation angle relative to the central axis of the surface satisfies the target central axis rotation amount as the intestinal feature points extracted from the supine three-dimensional surface and the prone three-dimensional surface.

[0010] In another embodiment, determining the first intestinal line and the second intestinal line corresponding to the supine three-dimensional surface and the prone three-dimensional surface based on the respective intestinal feature points includes: connecting the shortest surface distance between the respective intestinal feature points and performing smoothing to determine the first intestinal line and the second intestinal line corresponding to the supine three-dimensional surface and the prone three-dimensional surface.

[0011] In another embodiment, the first intestinal belt line and the second intestinal belt line each include three lines, and the first intestinal belt line and the second intestinal belt line are aligned to achieve colorectal surface alignment, including: calculating the target feature value between any one of the first intestinal belt lines and any one of the second intestinal belt lines; and selecting a pair of intestinal belt lines corresponding to the smallest target feature value for alignment to achieve colorectal surface alignment.

[0012] In yet another embodiment, the target characteristic value at least includes a curvature characteristic value.

[0013] In yet another embodiment, the method further includes: constructing a differential homeomorphism function; and optimizing the registration result of the colorectal surface registration based on the differential homeomorphism function.

[0014] In a second aspect, the present application provides a device for colorectal surface registration, comprising: a processor; and a memory on which computer instructions for colorectal surface registration are stored, and when the computer instructions are executed by the processor, multiple embodiments of the aforementioned first aspect are implemented.

[0015] In a third aspect, the present application provides a computer-readable storage medium having stored thereon computer program instructions for colorectal curved surface registration, wherein when the computer program instructions are executed by one or more processors, multiple embodiments of the aforementioned first aspect are implemented.

[0016] Through the solution for colorectal surface registration provided above, the embodiment of the present application performs colorectal surface registration by parameterizing the supine three-dimensional surface and the prone three-dimensional surface to the corresponding plane domain, extracting the intestinal feature points in the plane domain and determining the corresponding intestinal lines in the supine three-dimensional surface and the prone three-dimensional surface. Among them, the intestinal line identifies one of the important structures of the colon. Therefore, the embodiment of the present application makes the surface registration simple and stable by mapping the three-dimensional surface to a two-dimensional plane and performing colorectal surface registration through important identification structures (intestinal lines). BRIEF DESCRIPTION OF THE DRAWINGS

[0017] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present application will become easy to understand. In the accompanying drawings, several embodiments of the present application are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein: Figure 1 is an exemplary flowchart showing a method for colorectal curved surface registration according to an embodiment of the present application; Figure 2 is an exemplary schematic diagram showing a supine three-dimensional curved surface and a prone three-dimensional curved surface of a colorectum according to an embodiment of the present application; Figure 3 is an exemplary schematic diagram showing a plane domain corresponding to a supine three-dimensional curved surface and a prone three-dimensional curved surface according to an embodiment of the present application; Figure 4 is an exemplary coordinate schematic diagram showing the central axis alignment of the supine three-dimensional curved surface and the prone three-dimensional curved surface before and after alignment according to an embodiment of the present application; Figure 5 is a schematic image showing the alignment of the central axes of the supine three-dimensional curved surface and the prone three-dimensional curved surface; Figure 6 is an exemplary schematic diagram showing a filtered central axis according to an embodiment of the present application; Figure 7 is an exemplary schematic diagram showing a height map according to an embodiment of the present application; Figure 8 is an exemplary schematic diagram showing the extraction of feature rows according to an embodiment of the present application; Fig. 9 is an exemplary schematic diagram showing intestinal feature points of a supine three-dimensional curved surface and three corresponding intestinal lines according to an embodiment of the present application; Fig.10 1 is a diagram showing a final gut line corresponding to the supine three-dimensional curved surface according to an embodiment of the present application; Fig.11 is an exemplary schematic diagram showing a supine three-dimensional curved surface and a prone three-dimensional curved surface after gut line registration according to an embodiment of the present application; Fig.12 is an exemplary flowchart showing the overall process for colorectal curved surface registration according to an embodiment of the present application; Fig.13 It is an exemplary structural block diagram showing a device for colorectal curved surface registration according to an embodiment of the present application. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0019] It should be understood that the terms "include" and "comprising" used in the specification and claims of the present application indicate the presence of described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.

[0020] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this application specification and claims, unless the context clearly indicates otherwise, the singular forms of "a", "an" and "the" are intended to include plural forms. It should also be further understood that the term "and / or" used in this application specification and claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0021] As used in this specification and claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0022] The specific implementation of the present application is described in detail below with reference to the accompanying drawings.

[0023] Figure 1 FIG. 1 is an exemplary flowchart showing a method 100 for colorectal curved surface registration according to an embodiment of the present application. Figure 1 As shown in , at step S101, the supine three-dimensional surface and prone three-dimensional surface of the colorectum are collected, and the supine three-dimensional surface and the prone three-dimensional surface are parameterized to corresponding plane domains respectively. In an implementation scenario, the supine three-dimensional surface and the prone three-dimensional surface of the colorectum can be collected and obtained by, for example, a CT scanning device, the supine three-dimensional surface is the three-dimensional surface of the colorectum scanned in a supine posture, and the prone three-dimensional surface is the three-dimensional surface of the colorectum scanned in a prone posture of the subject. In some embodiments, the supine three-dimensional surface and the prone three-dimensional surface of the colorectum can be obtained by three-dimensionally reconstructing the supine and prone CT data collected by the CT scanning device. Among them, the aforementioned three-dimensional reconstruction algorithm may include a method based on multi-view geometry, a method based on deep learning, and the like.

[0024] Based on the collected supine three-dimensional surface and prone three-dimensional surface of the colorectum, they are first parameterized to the corresponding plane domains. It can be understood that parameterization is the process of unfolding a three-dimensional surface to a two-dimensional plane domain, and the two-dimensional plane domain includes but is not limited to a rectangular domain, a disc domain, etc. Preferably, the embodiment of the present application parameterizes the supine three-dimensional surface and the prone three-dimensional surface to a rectangular area respectively. In some embodiments, parameterization can be performed by methods such as conformal mapping or area-preserving mapping. For example, taking conformal mapping as an example, a hole is made in the entire intestine in the prone surface or the supine surface at the anus and the cecum, and then a cut (or closed curve) is calculated, and then the intestinal wall is cut along the cut (or closed curve), and spread to the rectangular area through conformal mapping. Based on this, by parameterizing the three-dimensional surface to a plane domain, subsequent calculations can be greatly simplified.

[0025] Next, at step S102, the intestinal feature points of the supine three-dimensional surface and the prone three-dimensional surface are extracted in the corresponding plane domain. In one embodiment, first, a height map corresponding to the supine three-dimensional surface and the prone three-dimensional surface is constructed in the corresponding plane domain. Among them, the height map represents the distance of the intestinal wall point on the supine three-dimensional surface or the prone three-dimensional surface from the central axis of the surface. Next, the respective feature rows are extracted from the height maps corresponding to the supine three-dimensional surface and the prone three-dimensional surface, so as to extract the intestinal feature points of the supine three-dimensional surface and the prone three-dimensional surface based on the respective feature rows. That is, the feature rows are first extracted from the height maps corresponding to the supine three-dimensional surface and the prone three-dimensional surface, and then the intestinal feature points are extracted.

[0026] In an implementation scenario, when constructing a height map, first determine the central axis of each of the supine three-dimensional surface and the prone three-dimensional surface. Specifically, multiple sets of sampling point sets can be collected along the longitudinal direction of the rectangular domains corresponding to the supine three-dimensional surface and the prone three-dimensional surface, and each set of sampling point sets contains multiple sampling points. Calculate the coordinate mean of each group of multiple sampling points, and the multiple sets of sampling point sets correspond to multiple coordinate mean points. Restore the multiple coordinate mean points to the supine three-dimensional surface or the prone three-dimensional surface for connection, and you can get the central axis of each of the supine three-dimensional surface and the prone three-dimensional surface. As an example, assume that 1000 sets of sampling point sets are collected along the longitudinal direction, that is, 1000 points are contained in the corresponding transverse direction. Each set of sampling point sets contains 100 sampling points, that is, each set of points contains 100 points in the longitudinal direction. By averaging the 100 longitudinal points of each group of point sets, 1000 groups correspond to 1000 coordinate mean points, and then restoring the 1000 coordinate mean points to the supine three-dimensional surface or the prone three-dimensional surface for connection, the central axis of the supine three-dimensional surface and the prone three-dimensional surface can be obtained.

[0027] In some embodiments, in order to improve the subsequent calculation accuracy, the embodiments of the present application also perform central axis registration on the central axis of the supine 3D curved surface and the prone 3D curved surface. The central axis registration can be achieved by using the length of the central axis as a feature, for example, using Dynamic Time Warping ("DTW").

[0028] After completing the above-mentioned central axis alignment, the intestines are preliminarily aligned longitudinally (i.e., along the direction of the central axis of the intestines), but there is still rotation between the two intestines. Therefore, the embodiment of the present application extracts the intestinal cord lines of the colorectum and aligns the intestinal cord lines to complete local rotation to improve the alignment accuracy. In some embodiments, the implementation of the present application also performs a filtering operation on the aligned central axis. In an implementation scenario, the rotation amount of the central axis (i.e., the rotation amount of the coordinate system around the tangent vector) is optimized until it meets the target rotation amount to filter the aligned central axis. Specifically, there is an active frame coordinate system for the central axis (i.e., a coordinate system with a point on the central axis as the origin), and the rotation amount of the coordinate system can be obtained by integration. As an example, a discrete method can be used to calculate the coordinate system of the points on the central axis, and the rotation angles of the two adjacent coordinate systems can be calculated to determine the rotation amount of the central axis. Since the initial central axis is relatively curved, its initial rotation amount must be greater than the target rotation amount (e.g., 25, i.e., 4 rotations). In this case, filtering is repeated on the central axis until the rotation amount is lower than the target rotation amount of 25.

[0029] It should be understood that the above-mentioned extraction of the central axis, alignment of the central axis, and filtering of the central axis are operations performed on the supine three-dimensional surface and the prone three-dimensional surface respectively, and the filtered central axis of the supine three-dimensional surface and the prone three-dimensional surface are obtained. For the above-mentioned filtered central axis, the height map corresponding to the supine three-dimensional surface and the prone three-dimensional surface can be constructed in the corresponding plane domain. In an implementation scenario, the distance from the intestinal wall point to the central axis is calculated by finding the point corresponding to the closest central axis for all points on the supine three-dimensional surface and the prone three-dimensional surface. In this scenario, a distance function, that is, the distance function from the point to the nearest central axis, can be constructed in the parameter space. Further, the distance function is subjected to, for example, Laplace filtering, and, for example, a Gabor filter is applied to enhance the differences in the horizontal and vertical directions in the distance function to obtain the height map of the supine three-dimensional surface and the prone three-dimensional surface, respectively, denoted as h(x,y), where x represents the coordinates of the horizontal sequence pixels and y represents the longitudinal pixel value (i.e., the height value). In some embodiments, the filter weight coefficients of the aforementioned Laplace filter can be calculated based on a generalized cross-validation method.

[0030] Next, extract the respective feature rows from the height maps corresponding to the supine three-dimensional surface and the prone three-dimensional surface. In one implementation scenario, the pixel values ​​of each row in the height maps corresponding to the supine three-dimensional surface and the prone three-dimensional surface are accumulated, and the local maximum value in the accumulated results of the height maps corresponding to the supine three-dimensional surface and the prone three-dimensional surface is calculated, so as to extract the feature rows of the supine three-dimensional surface and the prone three-dimensional surface based on the points of the local maximum value. Among them, by accumulating the pixel values ​​of each row, the influence of noise can be removed to ensure the accuracy of subsequent calculations. After accumulation, a one-dimensional data can be obtained, which is recorded as Since not every row of pixels has intestinal feature points, the embodiment of the present application extracts the local maximum value in the accumulation result (ie, considering the maximum value point of each row height and sequence pixels) to extract more significant rows as feature rows.

[0031] After obtaining the feature rows, the intestinal feature points of the supine three-dimensional surface and the prone three-dimensional surface are extracted based on the respective feature rows. In an implementation scenario, the target point whose local minimum value is less than the preset threshold is extracted from the feature row, and the point whose rotation angle relative to the central axis of the surface satisfies the target central axis rotation amount is determined as the intestinal feature point extracted from the supine three-dimensional surface and the prone three-dimensional surface. In some embodiments, the aforementioned preset threshold can be, for example, 20% of the x value. Specifically, the brightness h of the feature row is taken as the y value, and the sequential pixels of the feature row are taken as the x value. The relative minimum value x (that is, the local minimum value x) of h(x) is obtained according to the information of h(x), and it is ensured that the minimum value is 20% smaller than the x value of the surrounding points to extract the target point.

[0032] It should be understood that the colon band is formed by the thickening of the longitudinal muscles of the intestinal wall, and includes three bands: the omental band, the mesentery band, and the independent band. Therefore, after the colon is divided into multiple circles, each circle (i.e., longitudinally) contains three target points (i.e., three minimum points). After the three minimum points of the first circle are selected, there are three connection methods according to the number of feature points. For example, target point 1 of the first circle can be connected to target points 1, 2, or 3 of the second circle. In this scenario, the points whose rotation angles relative to the central axis of the surface satisfy the target central axis rotation amount (e.g., 25) are determined as the supine three-dimensional surface and the prone three-dimensional surface to extract the intestinal band feature points of each.

[0033] Based on the intestinal feature points extracted above, at step S103, the first intestinal line and the second intestinal line corresponding to the supine three-dimensional surface and the prone three-dimensional surface are determined based on the respective intestinal feature points. In an implementation scenario, the first intestinal line and the second intestinal line corresponding to the supine three-dimensional surface and the prone three-dimensional surface are determined by connecting the shortest surface distances between the respective intestinal feature points and performing smoothing. Specifically, the shortest surface distances between the respective intestinal feature points can be connected and smoothed, for example, by using a geodesic algorithm.

[0034] Further, at step S104, the first intestinal line and the second intestinal line are aligned to achieve colorectal surface alignment. In an implementation scenario, the target feature value between any one of the first intestinal lines and any one of the second intestinal lines is calculated, and then a pair of intestinal lines corresponding to the smallest target feature value is selected for alignment to achieve colorectal surface alignment. According to the foregoing, the colonic band includes three bands: the omental band, the mesenteric band, and the independent band. Therefore, the first intestinal line and the second intestinal line each include three bands, and the three intestinal lines need to be matched. There are three matching methods. As an example, assuming that the first intestinal line includes intestinal lines 1, 2, and 3, and the second intestinal line includes intestinal lines 1, 2, and 3, there are three schemes: 1-1, 2-2, and 3-3; 1-2, 2-3, and 3-1; and 1-3, 2-1, and 3-2.

[0035] In some embodiments, the aforementioned target feature value may include but is not limited to a curvature feature value, a length feature value, etc. It can be understood that if two gut lines correspond to each other, their bends are relatively close. Therefore, by calculating the similarity between any gut lines based on the target feature value, and calculating the total energy between the registrations, the scheme with the smallest total registration energy is selected as the final registration result. For example, in an exemplary scenario, the curvature feature value difference between any gut lines is calculated based on the target feature value being the curvature feature value, and a group of gut lines with the smallest sum of curvature feature value differences are taken as the registration result. For example, in the above three matching schemes (1-1, 2-2, 3-3; 1-2, 2-3, 3-1 and 1-3, 2-1, 3-2), the curvature feature value difference between the gut lines in different matching schemes is calculated respectively, assuming that the sum of the curvature feature value difference of the 1-1, 2-2, 3-3 schemes is the smallest, and the 1-1, 2-2, 3-3 schemes are taken as the matching results. Similarly, if the sum of the differences in curvature eigenvalues ​​of the 1-2, 2-3, and 3-1 solutions is the smallest, the 1-2, 2-3, and 3-1 solutions are taken as matching results.

[0036] In combination with the above description, it can be known that the embodiment of the present application performs colorectal surface registration by parameterizing the supine three-dimensional surface and the prone three-dimensional surface to the corresponding plane domain, extracting the intestinal feature points in the plane domain and determining the corresponding intestinal lines in the supine three-dimensional surface and the prone three-dimensional surface. Among them, the intestinal line identifies one of the important structures of the colon. Therefore, the embodiment of the present application maps the three-dimensional surface to a two-dimensional plane, and performs colorectal surface registration through important identification structures (intestinal lines), so that the surface registration has simplicity, stability and accuracy.

[0037] Figure 2 2 is an exemplary schematic diagram showing a supine three-dimensional curved surface and a prone three-dimensional curved surface of the colorectum according to an embodiment of the present application. Figure 2 Figure (a) shows the three-dimensional surface of the colorectum in supine position. Figure 2 FIG. (b) shows a three-dimensional curved surface of the colorectum in the prone position. In some embodiments, the three-dimensional curved surface of the colorectum in the prone position and the prone position can be obtained by performing three-dimensional reconstruction on the CT data in the prone position and the prone position acquired by the CT scanning device.

[0038] Figure 3 2 is an exemplary schematic diagram showing the plane domains corresponding to the supine three-dimensional curved surface and the prone three-dimensional curved surface according to an embodiment of the present application. Figure 3 Figure (a) shows the plane domain corresponding to the three-dimensional surface in the upward position, as shown in Figure 3 FIG. (b) shows the plane domain corresponding to the three-dimensional surface in the prone position. In some embodiments, the three-dimensional surface in the prone position and the three-dimensional surface in the prone position can be parameterized to the corresponding plane domain by using methods such as conformal mapping or area-preserving mapping. Preferably, the plane domain is a rectangular area. By parameterizing the three-dimensional surface to a plane domain, subsequent calculations can be greatly simplified.

[0039] After the supine 3D surface and the prone 3D surface are parameterized to the corresponding plane domain, their respective central axes can be extracted first. In some embodiments, multiple groups of sampling point sets containing multiple sampling points can be collected along the longitudinal direction of the rectangular domain corresponding to each of the supine 3D surface and the prone 3D surface, and the corresponding coordinate mean points in each group of sampling point sets are restored to the supine 3D surface or the prone 3D surface for connection, so as to obtain the central axes of the supine 3D surface and the prone 3D surface. Based on the extracted central axis, it can also be aligned using, for example, the DTW method to improve the accuracy of subsequent calculations.

[0040] Figure 4 2 is a schematic diagram showing exemplary coordinates of the supine three-dimensional curved surface and the prone three-dimensional curved surface before and after the central axis alignment according to an embodiment of the present application. Figure 4 The upper middle portion shows a schematic diagram of coordinates before the central axis alignment of the supine 3D curved surface and the prone 3D curved surface. Figure 4The coordinate diagram of the three-dimensional surface in the supine position and the three-dimensional surface in the prone position after the alignment of their respective central axes is shown in the lower middle part. The horizontal axis represents the sequence pixel coordinates, the vertical axis represents the height information, and the acupuncture position in the figure represents the intestinal loop. A and B in the figure correspond to the coordinate information curves of the central axes of the three-dimensional surface in the supine position and the three-dimensional surface in the prone position, respectively.

[0041] Figure 5 is a schematic image showing the alignment of the central axes of the supine 3D curved surface and the prone 3D curved surface. Figure 5 As shown in , L1 and L2 correspond to the central axis of the supine three-dimensional curved surface and the prone three-dimensional curved surface, respectively. In some embodiments, a filtering operation can also be performed on the aligned central axis to complete the intestinal rotation (for example Figure 5 Local registration of the part (shown in the circle in the middle).

[0042] Figure 6 is an exemplary schematic diagram showing the filtered central axis according to an embodiment of the present application. Figure 6 , which is a schematic diagram of the filtered central axis. In some embodiments, a discrete method can be used to calculate the coordinate system of the points on the central axis, and the rotation angles of two adjacent coordinate systems can be calculated to determine the rotation amount of the central axis, and the central axis is repeatedly filtered until the rotation amount is lower than the target rotation amount (e.g., 25), to obtain the filtered central axis. Based on the filtered central axis of each of the supine three-dimensional surface and the prone three-dimensional surface, by calculating the distance from the intestinal wall point to the central axis, the height map corresponding to each of the supine three-dimensional surface and the prone three-dimensional surface can be constructed.

[0043] Figure 7 is an exemplary schematic diagram showing a height map according to an embodiment of the present application. Figure 7 Shown in is a partial height map corresponding to the supine three-dimensional surface. As mentioned above, by finding the point corresponding to the closest central axis for all points on the intestinal wall on the supine three-dimensional surface, a distance function from the point to the nearest point on the central axis is constructed. Then, by, for example, performing a Laplace filter on the distance function, and applying, for example, a Gabor filter to enhance the differences in the horizontal and vertical directions in the distance function, the height map is obtained. Similarly, a height map corresponding to the prone three-dimensional surface can be obtained. Then, the respective feature rows are extracted from the height maps corresponding to the supine three-dimensional surface and the prone three-dimensional surface. Specifically, each row of the height map is accumulated to obtain a one-dimensional data. , by extracting the local maximum value, to obtain the feature row.

[0044] Figure 8 FIG. 1 is an exemplary schematic diagram showing the extraction of feature lines according to an embodiment of the present application. Figure 8The red circle shown in is the local maximum value, which corresponds to the feature line. Among them, the horizontal axis represents the sequence pixel coordinates, and the vertical axis represents the height information. After obtaining the feature line, the supine three-dimensional surface and the prone three-dimensional surface are extracted based on their respective feature lines to extract the respective intestinal feature points. In an implementation scenario, first, the points whose local minimum values ​​are less than 20% of their surroundings are extracted from the feature line as the target points, and the target points whose rotation angles relative to the central axis of the surface meet, for example, a rotation amount of 25 are determined as the supine three-dimensional surface and the prone three-dimensional surface to extract the respective intestinal feature points. Furthermore, intestinal lines can be obtained by connecting the intestinal feature points. The supine three-dimensional surface and the prone three-dimensional surface each correspond to three intestinal lines.

[0045] Fig. 9 FIG. 2 is an exemplary schematic diagram showing the intestinal feature points of the supine three-dimensional curved surface and the corresponding three intestinal lines according to an embodiment of the present application. Fig. 9 The small circles shown in the figure represent the intestinal feature points. Intestinal lines can be obtained by connecting the intestinal feature points, including three intestinal lines: the omental band, the mesenteric band, and the independent band (as shown by the red, green, and blue lines in the figure). In some embodiments, the present application also determines the final intestinal lines corresponding to the supine three-dimensional curved surface and the prone three-dimensional curved surface by connecting the shortest distances between the respective intestinal feature points and performing smoothing, for example Fig.10 shown.

[0046] Fig.10 : is a diagram showing the final gut line corresponding to the supine three-dimensional curved surface according to an embodiment of the present application. Fig.10 The final gut line (i.e., the first gut line) corresponding to the supine three-dimensional surface is shown. Similarly, the final gut line (i.e., the second gut line) corresponding to the prone three-dimensional surface can be obtained. The first gut line and the second gut line are registered to achieve colorectal surface registration. In an implementation scenario, the curvature eigenvalue differences between any gut lines are calculated respectively, and the group of gut lines with the smallest sum of curvature eigenvalue differences is taken as the registration result.

[0047] Fig.11 2 is an exemplary schematic diagram showing a supine three-dimensional curved surface and a prone three-dimensional curved surface after gut line registration according to an embodiment of the present application. Fig.11 Figure (a) shows the supine 3D surface after gut line registration. Fig.11 Figure (b) shows the three-dimensional surface in the prone position after the intestinal line is registered. In some embodiments, a differential homeomorphism function can also be constructed to optimize the registration result of the colorectal surface registration based on the differential homeomorphism function. It can be understood that a differential homeomorphism function refers to a function that satisfies a one-to-one mapping (i.e., satisfies both single injection and surjection at the same time) and is continuous and infinitely differentiable.

[0048] In an implementation scenario, first, a gut line (for example, a gut line in the supine 3D surface) can be selected as reference information and recorded as , and record the remaining gut line information (such as the gut line in the prone three-dimensional surface) as , the above registration result is recorded as In this scenario, the target registration function can be obtained = ,in represents the gradient of the reference information. Next, the registration function is updated based on the target registration function : (1) in, represents the weight, Indicates reference information. Indicates the rest of the gut line information, represents a constant coefficient. In this scenario, by transforming the target registration function Substituting into the above formula (1), we can obtain the updated registration function, which can be expressed as Further, the initial Beltrami coefficients are calculated based on the updated registration function. In one implementation scenario, the initial Beltrami coefficients can be calculated based on the interpolated registration function by performing linear interpolation on the registration function. Specifically, the initial Beltrami coefficients It can be expressed as follows: (2) in, , , and , , , Represents the coordinate information of the triangle vertices in the reference information (i.e., the vertices of the surface mesh in the colon surface), , , Represents the coordinate information of the triangle vertices below other gut lines, , , The barycentric coefficient of a triangle is expressed as a ratio of the area of ​​the triangle. For example, , represents the center of gravity. Similarly, we can obtain and Therefore, the aforementioned It is known that by Substituting into the above formula (2), the initial Beltrami coefficient can be obtained.

[0049] Further, the final registration function is obtained according to the initial Beltrami coefficients and the target registration function. In one embodiment, the initial Beltrami coefficients are first corrected so that the final registration function satisfies the differential homeomorphism (i.e., a differential homeomorphism function is obtained), and then the target registration function is reconstructed according to the corrected Beltrami coefficients, and then the final registration function is determined based on the reconstructed registration function and the target registration function. In an implementation scenario, the initial Beltrami coefficients can be corrected based on the following formula so that the final registration function satisfies the differential homeomorphism: (3) in represents the modified Beltrami coefficient, represents the Beltrami coefficient before correction (for example, the initial Beltrami coefficient ). According to the modified Beltrami coefficients, the target registration function can be reconstructed. In an exemplary scenario, assuming that the modified Beltrami coefficients , based on the above formula (1), the following formula can be obtained: (4) By rearranging formula (4), we can obtain:

[0050] in, , , , , substituting the above parameters into formula (5) and formula (6) we can obtain , thus the target registration function can be reconstructed and the reconstructed registration function can be obtained Next, a final registration function is determined based on the reconstructed registration function and the target registration function, and the final registration function is a differential homeomorphism function.

[0051] In one embodiment, in response to the maximum absolute value of the difference between the reconstructed registration function and the target registration function meeting a preset threshold, the reconstructed registration function is used as the final registration function. Specifically, when the maximum absolute value of the difference between the reconstructed registration function and the final registration function is less than the preset threshold, the reconstructed registration function is used as the final registration function. As an example, assuming that the target registration function is recorded as , the reconstructed registration function is recorded as , the preset threshold is recorded as , the reconstructed registration function Registration function with target The maximum absolute value of the difference between ,but .when < When the reconstructed registration function As the final registration function. On the contrary, when ≥ When using the reconstructed registration function Update the registration function (i.e., use the reconstructed registration function As the target registration function of the next iteration), repeat the above operations, including calculating the Beltrami coefficient based on the updated registration function, correcting the Beltrami coefficient, reconstructing the registration function, and calculating the maximum absolute value of the difference between the current registration function and the previous (or last iteration) registration function. , until < The target registration function (i.e., differential homeomorphism function) is obtained to further refine the registration result of colorectal surface registration based on the differential homeomorphism function.

[0052] Fig.12 FIG. 1 is an exemplary flowchart showing the overall process for colorectal curved surface registration according to an embodiment of the present application. Fig.12 As shown in , at step S1101 and step S1102, the supine three-dimensional surface and prone three-dimensional surface of the colorectum are collected respectively, and the supine three-dimensional surface and prone three-dimensional surface are parameterized to corresponding plane domains respectively. Then, at step S1103 and step S1104, the central axis of the supine three-dimensional surface and the prone three-dimensional surface are extracted respectively, and at step S1105, the central axis is aligned and filtered. Based on the filtered central axis, at step S1106 and step S1107, the height maps corresponding to the supine three-dimensional surface and the prone three-dimensional surface are constructed respectively.

[0053] Furthermore, at step S1108 and step S1109, the characteristic lines and intestinal feature points of the supine three-dimensional surface and the prone three-dimensional surface are extracted respectively. After the intestinal feature points are extracted, at step S1110 and step S1111, the first intestinal line and the second intestinal line corresponding to the supine three-dimensional surface and the prone three-dimensional surface are obtained based on the intestinal feature points. Then, at step S1112, the first intestinal line and the second intestinal line are aligned to achieve colorectal surface alignment. For more details on colorectal surface alignment, please refer to the above Figure 1 The description of is not repeated here in this application.

[0054] Fig.131 is an exemplary structural block diagram of a device 1200 for colorectal curved surface registration according to an embodiment of the present application. It is understood that the device 1200 may include the apparatus of the embodiment of the present application, and the device implementing the solution of the present application may be a single device (such as a computing device) or a multifunctional device including various peripheral devices.

[0055] like Fig.13 As shown in , the device of the present application may also include a central processing unit or central processing unit ("CPU") 1211, which may be a general-purpose CPU, a dedicated CPU, or other information processing and program execution unit. Further, the device 1200 may also include a large-capacity memory 1212 and a read-only memory ("ROM") 1213, wherein the large-capacity memory 1212 may be configured to store various types of data, including various supine three-dimensional surfaces and prone three-dimensional surfaces related to the colorectum, the central axis, the height map, the intestinal band line, the algorithm data, the intermediate results, and various programs required to run the device 1200. The ROM 1213 may be configured to store the power-on self-test of the device 1200, the initialization of each functional module in the system, the basic input / output driver of the system, and the data and instructions required to boot the operating system.

[0056] Optionally, the device 1200 may also include other hardware platforms or components, such as the tensor processing unit ("TPU") 1214, the graphics processing unit ("GPU") 1215, the field programmable gate array ("FPGA") 1216, and the machine learning unit ("MLU") 1217 shown. It is understood that although a variety of hardware platforms or components are shown in the device 1200, they are only exemplary and not restrictive, and those skilled in the art can add or remove corresponding hardware according to actual needs. For example, the device 1200 may only include a CPU, related storage devices, and interface devices to implement the method for colorectal surface registration of the present application.

[0057] In some embodiments, in order to facilitate the transmission and interaction of data with an external network, the device 1200 of the present application further includes a communication interface 1218, so that it can be connected to a local area network / wireless local area network ("LAN / WLAN") 1205 through the communication interface 1218, and then connected to a local server 1206 or to the Internet ("Internet") 1207 through the LAN / WLAN. Alternatively or additionally, the device 1200 of the present application can also be directly connected to the Internet or a cellular network through the communication interface 1218 based on wireless communication technology, such as wireless communication technology based on the third generation ("3G"), the fourth generation ("4G") or the fifth generation ("5G") . In some application scenarios, the device 1200 of the present application can also access the server 1208 and the database 1209 of the external network as needed to obtain various known algorithms, data and modules, and can remotely store various data, such as various types of data or instructions for presenting, for example, three-dimensional curved surfaces in supine and prone positions, central axis lines, height maps, intestinal band lines, etc. of the colorectum.

[0058] The peripheral devices of the device 1200 may include a display device 1202, an input device 1203, and a data transmission interface 1204. In one embodiment, the display device 1202 may include, for example, one or more speakers and / or one or more visual displays, which are configured to perform voice prompts and / or image video display for the colorectal surface registration of the present application. The input device 1203 may include other input buttons or controls such as a keyboard, a mouse, a microphone, a gesture capture camera, etc., which are configured to receive input of audio data and / or user instructions. The data transmission interface 1204 may include, for example, a serial interface, a parallel interface or a universal serial bus interface ("USB"), a small computer system interface ("SCSI"), a serial ATA, a FireWire ("FireWire"), a PCI Express, and a high-definition multimedia interface ("HDMI"), etc., which are configured for data transmission and interaction with other devices or systems. According to the solution of the present application, the data transmission interface 1204 can receive the supine three-dimensional surface and prone three-dimensional surface of the colorectum acquired from the CT device, and transmit the supine three-dimensional surface and prone three-dimensional surface of the colorectum or various other types of data or results to the device 1200.

[0059] The CPU 1211, mass storage 1212, ROM 1213, TPU 1214, GPU 1215, FPGA 1216, MLU 1217 and communication interface 1218 of the device 1200 of the present application can be interconnected through a bus 1219, and data interaction with peripheral devices is achieved through the bus. In one embodiment, through the bus 1219, the CPU 1211 can control other hardware components in the device 1200 and its peripheral devices.

[0060] Combination of the above Fig.13 The device for colorectal curved surface registration that can be used to perform the present application is described. It should be understood that the device structure or architecture here is only exemplary, and the implementation method and implementation entity of the present application are not limited thereto, but can be changed without departing from the spirit of the present application.

[0061] According to the above description in combination with the accompanying drawings, those skilled in the art can also understand that the embodiments of the present application can also be implemented by software programs. Therefore, the present application also provides a computer-readable storage medium, on which computer-readable instructions for colorectal surface registration are stored. When the computer-readable instructions are executed by one or more processors, they can be used to implement the present application in combination with the accompanying drawings. Figure 1 , Fig.12 The described method for colorectal surface registration.

[0062] It should be noted that although the operations of the method of the present application are described in a specific order in the accompanying drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. On the contrary, the steps depicted in the flow chart can be performed in a different order. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step, and / or one step can be decomposed into multiple steps.

[0063] It should be understood that when the terms "first", "second", "third" and "fourth" are used in the claims, the specification and the drawings of the present application, they are only used to distinguish different objects, rather than to describe a specific order. The terms "include" and "comprise" used in the specification and claims of the present application indicate the presence of the described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or their collections.

[0064] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this application specification and claims, unless the context clearly indicates otherwise, the singular forms of "a", "an" and "the" are intended to include plural forms. It should also be further understood that the term "and / or" used in this application specification and claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0065] Although the implementation methods of the present application are as above, the contents described are only examples adopted to facilitate the understanding of the present application, and are not intended to limit the scope and application scenarios of the present application. Any technician in the technical field described in the present application can make any modifications and changes in the form and details of the implementation without departing from the spirit and scope disclosed in the present application, but the scope of patent protection of the present application shall still be subject to the scope defined in the attached claims.

[0066] In addition, the collection and acquisition of various data in this application complies with relevant laws and regulations and is authorized by the data provider. Any organization or individual that needs to obtain external data must obtain authorization in accordance with the law and ensure data security. It is not allowed to illegally collect, use, process, or transmit unauthorized or unprotected data, or to illegally buy, sell, provide, or disclose unauthorized or unprotected data.

Claims

1. A method for colorectal surface registration, characterized in that: include: Collecting a supine three-dimensional curved surface and a prone three-dimensional curved surface of the colorectum, and parameterizing the supine three-dimensional curved surface and the prone three-dimensional curved surface to corresponding plane domains respectively; Extracting intestinal feature points of the supine three-dimensional curved surface and the prone three-dimensional curved surface respectively in the corresponding plane domain; Determine a first gut line and a second gut line corresponding to the supine three-dimensional curved surface and the prone three-dimensional curved surface based on the respective gut feature points; The first intestinal belt line and the second intestinal belt line are registered to achieve colorectal curved surface registration, The step of extracting the intestinal feature points of the supine three-dimensional curved surface and the prone three-dimensional curved surface in the corresponding plane domain comprises: Constructing a height map corresponding to each of the supine three-dimensional curved surface and the prone three-dimensional curved surface in a corresponding plane domain, wherein the height map represents the distance between an intestinal wall point on the supine three-dimensional curved surface or the prone three-dimensional curved surface and the central axis of the curved surface; Accumulating pixel values ​​of each row in the height map corresponding to the supine three-dimensional curved surface and the prone three-dimensional curved surface; Calculate the local maximum value in the accumulated results of the height maps corresponding to the supine three-dimensional curved surface and the prone three-dimensional curved surface; Extracting characteristic lines of the supine three-dimensional curved surface and the prone three-dimensional curved surface respectively based on the points of the local maximum values; Based on the respective feature lines, the intestinal feature points of the supine three-dimensional curved surface and the prone three-dimensional curved surface are extracted.

2. The method according to claim 1, characterized in that Extracting the intestinal feature points of the supine three-dimensional curved surface and the prone three-dimensional curved surface based on the feature line comprises: Extracting target points whose local minimum values ​​are less than a preset threshold value from the feature rows; The points of the target points whose rotation angles relative to the central axis of the curved surface satisfy the target central axis rotation amount are determined as the intestinal feature points of the supine three-dimensional curved surface and the prone three-dimensional curved surface to extract their respective ones.

3. The method according to claim 1, characterized in that Wherein determining the first intestinal line and the second intestinal line corresponding to the supine three-dimensional curved surface and the prone three-dimensional curved surface based on the respective intestinal feature points comprises: The shortest distances of the curved surfaces between the respective intestinal feature points are connected and smoothed to determine the first intestinal line and the second intestinal line corresponding to the supine three-dimensional curved surface and the prone three-dimensional curved surface.

4. The method according to claim 1 or 3, characterized in that: The first intestinal belt line and the second intestinal belt line each include three, and registering the first intestinal belt line and the second intestinal belt line to achieve colorectal curved surface registration includes: Calculating a target feature value between any one of the first gut lines and any one of the second gut lines; A pair of intestinal lines corresponding to the smallest target eigenvalue is selected for registration to achieve colorectal surface registration.

5. The method according to claim 4, characterized in that The target characteristic value at least includes a curvature characteristic value.

6. The method according to claim 4, characterized in that It also includes: Constructing diffeomorphic functions; The registration result of colorectal surface registration is optimized based on the diffeomorphism function.

7. A device for colorectal curved surface registration, characterized in that: include: processor; A memory having computer instructions for colorectal surface registration stored thereon, which, when executed by a processor, implement the method according to any one of claims 1-6.

8. A computer-readable storage medium, characterized in that: Computer program instructions for colorectal surface registration are stored thereon, and when the computer program instructions are executed by one or more processors, the method according to any one of claims 1-6 is implemented.

Images