Method, device and storage medium for colorectal surface registration

Abstract

The present application discloses a method, device and storage medium for colorectal surface registration. The method includes: collecting the supine three-dimensional surface and prone three-dimensional surface of the colorectum, and parameterizing the supine three-dimensional surface and the prone three-dimensional surface to corresponding plane domains respectively; extracting the intestinal feature points of each of the supine three-dimensional surface and the prone three-dimensional surface in the corresponding plane domain; determining the first intestinal line and the second intestinal line corresponding to the supine three-dimensional surface and the prone three-dimensional surface based on the respective intestinal feature points; and aligning the first intestinal line and the second intestinal line to achieve colorectal surface registration. Utilizing the solution of the present application, an accurate and stable registration method can be provided to align the supine three-dimensional surface and the prone three-dimensional surface of the colorectum.

Description

Technical Field

[0001] The present invention generally relates to the field of image processing technology, and more particularly to a method, device, and computer storage medium for colorectal curved surface registration. Background Art

[0002] Colorectal cancer is currently the second leading cause of cancer-related death. The development of polyps is often a precursor to colorectal cancer. Early detection, intervention, and removal of polyps can significantly reduce lesion mortality. Traditional optical colonoscopy allows for the observation and removal of polyps, and has therefore gained widespread use. However, optical colonoscopy requires general anesthesia and is invasive and destructive. Furthermore, by the time patients undergo optical colonoscopy, many are already past the optimal intervention period, making it unsuitable for large-scale screening.

[0003] The virtual colonoscopy solution based on Computer Aided Diagnosis (CAD) is currently the recommended method for large-scale screening. It is usually scanned using CT equipment and is therefore non-invasive. By analyzing and processing the three-dimensional colon surfaces acquired by the CT equipment, colon polyps can be effectively found and early warnings can be given. For virtual colonoscopy systems, surface registration can facilitate comparison of unclear areas or confirm the location of polyps. However, when scanning with a CT device, although the inner wall of the intestine has been cleaned, there are still cleaning residues. It is usually necessary to scan the colon surfaces at multiple angles (such as prone position, supine position, etc.) with a CT device for registration to confirm the location of the polyps. Therefore, how to align the colon surfaces at multiple angles has become a technical problem that needs to be solved urgently.

[0004] In view of this, there is an urgent need to provide a solution for colorectal surface registration. By parameterizing the supine three-dimensional surface and the prone three-dimensional surface to the corresponding two-dimensional plane domain, the computational complexity of the three-dimensional space is simplified. Then, the colorectal surface is registered within the two-dimensional plane domain through the important and easily identifiable structural intestinal lines, so as to provide an accurate and stable registration method to align the supine three-dimensional surface and the prone three-dimensional surface of the colorectum. Summary of the Invention

[0005] In order to at least solve one or more of the technical problems mentioned above, the present application proposes a solution for colorectal curved surface registration in multiple aspects.

[0006] In a first aspect, the present application provides a method for colorectal surface registration, comprising: collecting a supine three-dimensional surface and a prone three-dimensional surface of the colorectum, and parameterizing the supine three-dimensional surface and the prone three-dimensional surface to corresponding plane domains, respectively; extracting the intestinal feature points of each of the supine three-dimensional surface and the prone three-dimensional surface within the corresponding plane domain; determining the first intestinal line and the second intestinal line corresponding to the supine three-dimensional surface and the prone three-dimensional surface based on the respective intestinal feature points; and aligning the first intestinal line and the second intestinal line to achieve colorectal surface registration.

[0007] In one embodiment, extracting the intestinal feature points of each of the supine three-dimensional surface and the prone three-dimensional surface in the corresponding plane domain includes: constructing a height map corresponding to each of the supine three-dimensional surface and the prone three-dimensional surface in the corresponding plane domain, wherein the height map represents the distance between the intestinal wall point on the supine three-dimensional surface or the prone three-dimensional surface and the central axis of the surface; extracting respective feature rows from the height maps corresponding to each of the supine three-dimensional surface and the prone three-dimensional surface; and extracting respective intestinal feature points of the supine three-dimensional surface and the prone three-dimensional surface based on the respective feature rows.

[0008] In another embodiment, extracting respective feature rows from the height maps corresponding to the supine three-dimensional curved surface and the prone three-dimensional curved surface includes: accumulating pixel values of each row in the height maps corresponding to the supine three-dimensional curved surface and the prone three-dimensional curved surface; calculating the local maximum value in the accumulated results in the height maps corresponding to the supine three-dimensional curved surface and the prone three-dimensional curved surface; and extracting respective feature rows of the supine three-dimensional curved surface and the prone three-dimensional curved surface based on the points of the local maximum value.

[0009] In another embodiment, the extraction of the intestinal feature points of the supine three-dimensional surface and the prone three-dimensional surface based on the feature row includes: extracting a target point whose local minimum value is less than a preset threshold from the feature row; and determining the point among the target points whose rotation angle relative to the central axis of the surface satisfies the target central axis rotation amount as the intestinal feature point extracted from the supine three-dimensional surface and the prone three-dimensional surface.

[0010] In another embodiment, determining the first intestinal line and the second intestinal line corresponding to the supine three-dimensional surface and the prone three-dimensional surface based on the respective intestinal feature points includes: connecting the shortest distance of the surfaces between the respective intestinal feature points and performing smoothing to determine the first intestinal line and the second intestinal line corresponding to the supine three-dimensional surface and the prone three-dimensional surface.

[0011] In another embodiment, the first intestinal belt line and the second intestinal belt line each include three, and aligning the first intestinal belt line and the second intestinal belt line to achieve colorectal surface alignment includes: calculating the target feature value between any one of the first intestinal belt lines and any one of the second intestinal belt lines; and selecting a pair of intestinal belt lines corresponding to the smallest target feature value for alignment to achieve colorectal surface alignment.

[0012] In yet another embodiment, the target characteristic value includes at least a curvature characteristic value.

[0013] In yet another embodiment, the method further includes: constructing a differential homeomorphism function; and optimizing the registration result of the colorectal surface registration based on the differential homeomorphism function.

[0014] In a second aspect, the present application provides a device for colorectal surface registration, comprising: a processor; and a memory on which computer instructions for colorectal surface registration are stored. When the computer instructions are executed by the processor, multiple embodiments of the aforementioned first aspect are implemented.

[0015] In a third aspect, the present application provides a computer-readable storage medium having stored thereon computer program instructions for colorectal curved surface registration, wherein when the computer program instructions are executed by one or more processors, multiple embodiments of the aforementioned first aspect are implemented.

[0016] Through the solution for colorectal surface registration provided above, the embodiment of the present application performs colorectal surface registration by parameterizing the supine three-dimensional surface and the prone three-dimensional surface to the corresponding plane domain, extracting the intestinal feature points in the plane domain and determining the corresponding intestinal lines in the supine three-dimensional surface and the prone three-dimensional surface. Among them, the intestinal line identifies one of the important structures of the colon. Therefore, the embodiment of the present application makes the surface registration simple and stable by mapping the three-dimensional surface to a two-dimensional plane and performing colorectal surface registration through the important identification structure (intestinal line). BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The above and other objects, features and advantages of the exemplary embodiments of the present application will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present application are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:

[0018] Figure 1 is an exemplary flowchart illustrating a method for colorectal curved surface registration according to an embodiment of the present application;

[0019] Figure 2is an exemplary schematic diagram showing a supine three-dimensional curved surface and a prone three-dimensional curved surface of the colorectum according to an embodiment of the present application;

[0020] Figure 3 is an exemplary schematic diagram showing plane domains corresponding to a supine three-dimensional curved surface and a prone three-dimensional curved surface according to an embodiment of the present application;

[0021] Figure 4 1 is a schematic diagram showing exemplary coordinates of a supine three-dimensional curved surface and a prone three-dimensional curved surface before and after alignment of their respective central axes according to an embodiment of the present application;

[0022] Figure 5 is a schematic image showing the alignment of the respective central axes of the supine three-dimensional curved surface and the prone three-dimensional curved surface;

[0023] Figure 6 is an exemplary schematic diagram showing a filtered central axis according to an embodiment of the present application;

[0024] Figure 7 is an exemplary schematic diagram showing a height map according to an embodiment of the present application;

[0025] Figure 8 is an exemplary schematic diagram illustrating extraction of feature rows according to an embodiment of the present application;

[0026] Figure 9 is an exemplary schematic diagram showing intestinal feature points and three corresponding intestinal lines of a supine three-dimensional curved surface according to an embodiment of the present application;

[0027] Figure 10 1 is a diagram showing a final gut line corresponding to the supine three-dimensional curved surface according to an embodiment of the present application;

[0028] Figure 11 is an exemplary schematic diagram showing a supine three-dimensional curved surface and a prone three-dimensional curved surface after gut line registration according to an embodiment of the present application;

[0029] Figure 12 is an exemplary flow chart showing the overall process for colorectal curved surface registration according to an embodiment of the present application;

[0030] Figure 13 1 is an exemplary structural block diagram showing a device for colorectal curved surface registration according to an embodiment of the present application. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0032] It should be understood that the terms "include" and "comprising" used in the description and claims of this application indicate the presence of described features, wholes, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or collections thereof.

[0033] It should also be understood that the terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit this application. As used in this specification and claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should also be further understood that the term "and / or" as used in this specification and claims refers to any and all possible combinations of one or more of the associated listed items, including and including these combinations.

[0034] As used in this specification and claims, the term “if” can be interpreted as “when” or “upon” or “in response to determining” or “in response to detecting,” depending on the context. Similarly, the phrase “if it is determined” or “if [described condition or event] is detected” can be interpreted as meaning “upon determination” or “in response to determining” or “upon detection of [described condition or event]” or “in response to detecting [described condition or event],” depending on the context.

[0035] The specific implementation of the present application will be described in detail below with reference to the accompanying drawings.

[0036] Figure 1 FIG. 1 is an exemplary flow chart showing a method 100 for colorectal curved surface registration according to an embodiment of the present application. Figure 1As shown in , at step S101, the supine three-dimensional surface and prone three-dimensional surface of the colorectum are collected, and the supine three-dimensional surface and prone three-dimensional surface are parameterized to corresponding plane domains respectively. In an implementation scenario, the supine three-dimensional surface and prone three-dimensional surface of the colorectum can be collected and obtained by, for example, a CT scanning device, the supine three-dimensional surface is the three-dimensional surface of the colorectum scanned in a supine posture, and the prone three-dimensional surface is the three-dimensional surface of the colorectum scanned in a prone posture. In some embodiments, the supine three-dimensional surface and prone three-dimensional surface of the colorectum can be obtained by performing three-dimensional reconstruction on the supine and prone CT data collected by the CT scanning device. Among them, the aforementioned three-dimensional reconstruction algorithm may include a method based on multi-view geometry, a method based on deep learning, and the like.

[0037] Based on the collected supine three-dimensional surface and prone three-dimensional surface of the colorectum, they are first parameterized to the corresponding plane domains. It can be understood that parameterization is the process of unfolding a three-dimensional surface to a two-dimensional plane domain, and the two-dimensional plane domain includes but is not limited to a rectangular domain, a disc domain, etc. Preferably, the embodiment of the present application parameterizes the supine three-dimensional surface and the prone three-dimensional surface to a rectangular area respectively. In some embodiments, parameterization can be performed using methods such as conformal mapping or area-preserving mapping. For example, taking conformal mapping as an example, a hole is made in the entire intestine in the prone surface or the supine surface at the anus and the cecum respectively, and then a cut (or closed curve) is calculated, and then the intestinal wall is cut along the cut (or closed curve), and spread out to a rectangular area through conformal mapping. Based on this, by parameterizing the three-dimensional surface to a plane domain, subsequent calculations can be greatly simplified.

[0038] Then, at step S102, the intestinal feature points of each of the supine three-dimensional surface and the prone three-dimensional surface are extracted in the corresponding plane domain. In one embodiment, first, a height map corresponding to each of the supine three-dimensional surface and the prone three-dimensional surface is constructed in the corresponding plane domain. The height map represents the distance between the intestinal wall point on the supine three-dimensional surface or the prone three-dimensional surface and the central axis of the surface. Then, the respective feature rows are extracted from the height maps corresponding to each of the supine three-dimensional surface and the prone three-dimensional surface, so as to extract the respective intestinal feature points of the supine three-dimensional surface and the prone three-dimensional surface based on the respective feature rows. That is, the feature rows are first extracted from the height maps corresponding to each of the supine three-dimensional surface and the prone three-dimensional surface, and then the intestinal feature points are extracted.

[0039] In an implementation scenario, when constructing a height map, first determine the central axis of each of the supine three-dimensional surface and the prone three-dimensional surface. Specifically, multiple sets of sampling point sets can be collected along the longitudinal direction of the rectangular domain corresponding to each of the supine three-dimensional surface and the prone three-dimensional surface, and each set of sampling point sets contains multiple sampling points. Calculate the coordinate mean of each set of multiple sampling points, and the multiple sets of sampling point sets correspond to multiple coordinate mean points. The multiple coordinate mean points are restored to the supine three-dimensional surface or the prone three-dimensional surface for connection, and the central axis of each of the supine three-dimensional surface and the prone three-dimensional surface can be obtained. As an example, assume that 1000 sets of sampling point sets are collected along the longitudinal direction, that is, 1000 points are contained in the horizontal direction. Each set of sampling point sets contains 100 sampling points, that is, each set of points contains 100 points in the longitudinal direction. By averaging the 100 longitudinal points of each group of point sets, 1000 groups correspond to 1000 coordinate mean points, and then restoring the 1000 coordinate mean points to the supine three-dimensional surface or the prone three-dimensional surface for connection, the central axis of the supine three-dimensional surface and the prone three-dimensional surface can be obtained.

[0040] In some embodiments, to improve subsequent calculation accuracy, embodiments of the present application further perform central axis registration on the central axes of the supine and prone 3D curved surfaces. This central axis registration can be achieved using, for example, dynamic time warping (DTW), using the length of the central axis as a feature.

[0041] After completing the above-mentioned central axis registration, the intestines are preliminarily aligned longitudinally (i.e., along the central axis of the intestines), but there is still rotation between the two intestines. Therefore, the embodiments of the present application extract the intestinal cord lines of the colorectal tract and align the intestinal cord lines to complete local rotation to improve the registration accuracy. In some embodiments, the implementation of the present application also performs a filtering operation on the aligned central axis. In one implementation scenario, the rotation amount of the central axis (i.e., the rotation amount of the coordinate system around the tangent vector) is optimized until it meets the target rotation amount, thereby filtering the aligned central axis. Specifically, the central axis has a moving frame coordinate system (i.e., a coordinate system with a point on the central axis as the origin), and the rotation amount of this coordinate system can be obtained by integration. As an example, a discrete method can be used to calculate the coordinate system of the points on the central axis, and the rotation angle of the two adjacent coordinate systems can be calculated to determine the rotation amount of the central axis. Since the initial central axis is relatively curved, its initial rotation amount must be greater than the target rotation amount (e.g., 25, i.e., 4 rotations). In this case, filtering is repeated on the central axis until the rotation amount is lower than the target rotation amount of 25.

[0042] It should be understood that the aforementioned extraction of the central axis, alignment of the central axis, and filtering of the central axis are performed separately on the supine and prone 3D surfaces, obtaining filtered central axes for each of the supine and prone 3D surfaces. Based on these filtered central axes, height maps corresponding to the supine and prone 3D surfaces can be constructed within the corresponding plane domains. In one implementation scenario, the distance from each intestinal wall point to the central axis is calculated by searching for the point corresponding to the closest central axis for all points on the supine and prone 3D surfaces. In this scenario, a distance function can be constructed within the parameter space, i.e., the distance function from a point to the nearest central axis. Furthermore, the distance function is subjected to, for example, a Laplacian filter, and a Gabor filter is applied to enhance the horizontal and vertical differences in the distance function to obtain height maps for each of the supine and prone 3D surfaces, denoted as h(x,y), where x represents the coordinate of the horizontal pixel sequence and y represents the vertical pixel value (i.e., the height value). In some embodiments, the filtering weight coefficients of the aforementioned Laplace filter can be calculated based on a generalized cross-validation method.

[0043] Next, extract the respective feature rows from the height maps corresponding to the supine three-dimensional surface and the prone three-dimensional surface. In one implementation scenario, by accumulating the pixel values of each row in the height maps corresponding to the supine three-dimensional surface and the prone three-dimensional surface, calculate the local maximum value in the accumulated results of the height maps corresponding to the supine three-dimensional surface and the prone three-dimensional surface, and extract the feature rows of each supine three-dimensional surface and the prone three-dimensional surface based on the points of the local maximum value. Among them, by accumulating the pixel values of each row, the influence of noise can be removed to ensure the accuracy of subsequent calculations. After accumulation, a one-dimensional data can be obtained, which is recorded as Since not every row of pixels has intestinal feature points, the embodiment of the present application extracts the local maximum value in the accumulated result (i.e., considering the maximum point of each row height and sequence pixels) to extract more significant rows as feature rows.

[0044] After obtaining the feature rows, the intestinal feature points of the supine three-dimensional surface and the prone three-dimensional surface are extracted based on their respective feature rows. In one implementation scenario, a target point whose local minimum value is less than a preset threshold is extracted from the feature row, and the point whose rotation angle relative to the central axis of the surface satisfies the target central axis rotation amount is determined as the intestinal feature point extracted from the supine three-dimensional surface and the prone three-dimensional surface. In some embodiments, the aforementioned preset threshold can be, for example, 20% of the x value. Specifically, the brightness h of the feature row is used as the y value, and the sequential pixels of the feature row are used as the x value. The relative minimum value x (that is, the local minimum value x) of h(x) is obtained based on the information of h(x), and it is ensured that the minimum value is 20% smaller than the x value of the surrounding points to extract the target point.

[0045] It should be understood that the colic band is formed by the thickening of the longitudinal muscles of the intestinal wall, and includes three bands: the omental band, the mesenteric band, and the independent band. Therefore, after the colon is divided into multiple circles, each circle (i.e., longitudinally) contains three target points (i.e., three minimum points). After the three minimum points of the first circle are selected, there are three connection methods according to the number of feature points. For example, target point 1 of the first circle can be connected to target points 1, 2, or 3 of the second circle. In this scenario, the points whose rotation angles relative to the central axis of the surface meet the target central axis rotation amount (e.g., 25) are determined as the supine three-dimensional surface and the prone three-dimensional surface to extract the intestinal band feature points of each.

[0046] Based on the extracted intestinal feature points, at step S103, first and second intestinal lines corresponding to the supine and prone three-dimensional curved surfaces are determined based on the respective intestinal feature points. In one implementation scenario, the first and second intestinal lines corresponding to the supine and prone three-dimensional curved surfaces are determined by connecting the shortest surface distances between the respective intestinal feature points and performing a smoothing process. Specifically, the shortest surface distances between the respective intestinal feature points can be connected and smoothed, for example, using a geodesic algorithm.

[0047] Furthermore, at step S104, the first intestinal line and the second intestinal line are aligned to achieve colorectal surface alignment. In an implementation scenario, the target feature value between any one of the first intestinal lines and any one of the second intestinal lines is calculated, and then a pair of intestinal lines corresponding to the smallest target feature value is selected for alignment to achieve colorectal surface alignment. As previously known, the colonic band includes three: the omental band, the mesenteric band, and the independent band. Therefore, the first intestinal line and the second intestinal line each include three, and the three intestinal lines need to be matched. There are three matching methods. As an example, assuming that the first intestinal line includes intestinal lines 1, 2, and 3, and the second intestinal line includes intestinal lines 1, 2, and 3, there are three schemes: 1-1, 2-2, and 3-3; 1-2, 2-3, and 3-1; and 1-3, 2-1, and 3-2.

[0048] In some embodiments, the aforementioned target feature values may include, but are not limited to, curvature feature values and length feature values. It will be understood that if two gut lines correspond, their curvatures are relatively close. Therefore, by calculating the similarity between any gut lines based on the target feature values and calculating the total energy between the registrations, the scheme with the lowest total energy is selected as the final registration result. For example, in an exemplary scenario, where the target feature value is the curvature feature value, the curvature feature value differences between any gut lines are calculated, and the group of gut lines with the smallest sum of curvature feature value differences is selected as the registration result. For example, in the three matching schemes described above (1-1, 2-2, 3-3; 1-2, 2-3, 3-1; and 1-3, 2-1, 3-2), the curvature feature value differences between the gut lines in each matching scheme are calculated. Assuming that the sum of the curvature feature value differences for schemes 1-1, 2-2, and 3-3 is the smallest, schemes 1-1, 2-2, and 3-3 are selected as the matching results. Similarly, if the sum of the differences in the curvature eigenvalues of the 1-2, 2-3, and 3-1 solutions is the smallest, the 1-2, 2-3, and 3-1 solutions are taken as matching results.

[0049] In combination with the above description, it can be seen that the embodiment of the present application performs colorectal surface registration by parameterizing the supine three-dimensional surface and the prone three-dimensional surface to the corresponding plane domain, extracting the intestinal feature points in the plane domain and determining the corresponding intestinal lines in the supine three-dimensional surface and the prone three-dimensional surface. Among them, the intestinal line identifies one of the important structures of the colon. Therefore, the embodiment of the present application maps the three-dimensional surface to a two-dimensional plane and performs colorectal surface registration through the important identification structure (intestinal line), so that the surface registration has simplicity, stability and accuracy.

[0050] Figure 2 1 is an exemplary schematic diagram showing the supine three-dimensional curved surface and prone three-dimensional curved surface of the colorectum according to an embodiment of the present application. Figure 2 Figure (a) shows the supine three-dimensional surface of the colorectum, as shown in Figure 2 FIG (b) shows a prone three-dimensional surface of the colorectum. In some embodiments, the prone three-dimensional surface and prone three-dimensional surface of the colorectum can be obtained by performing three-dimensional reconstruction on the prone and supine CT data acquired by a CT scanning device.

[0051] Figure 3 : is an exemplary schematic diagram showing the plane domains corresponding to the supine three-dimensional curved surface and the prone three-dimensional curved surface according to an embodiment of the present application. Figure 3 Figure (a) shows the plane domain corresponding to the three-dimensional surface in the upward position, as shown in Figure 3Figure (b) shows the planar domain corresponding to the prone 3D surface. In some embodiments, parameterization methods such as conformal mapping or area-preserving mapping can be used to parameterize the prone and prone 3D surfaces into corresponding planar domains. Preferably, the planar domains are rectangular regions. Parameterizing 3D surfaces into planar domains greatly simplifies subsequent calculations.

[0052] After the supine and prone 3D surfaces are parameterized to corresponding planar domains, their respective central axes can first be extracted. In some embodiments, multiple sets of sampling points containing multiple sampling points can be collected longitudinally along the rectangular domains corresponding to the supine and prone 3D surfaces, respectively. The corresponding coordinate mean points of each set of sampling points are restored to the supine or prone 3D surfaces and connected to obtain the central axes of each of the supine and prone 3D surfaces. Based on the extracted central axes, they can also be aligned using methods such as DTW to improve the accuracy of subsequent calculations.

[0053] Figure 4 : is an exemplary coordinate diagram showing the alignment of the central axis of the supine three-dimensional curved surface and the prone three-dimensional curved surface according to an embodiment of the present application. Figure 4 The upper middle portion shows a coordinate diagram of the supine 3D surface and the prone 3D surface before their respective central axes are aligned. Figure 4 The lower center shows a coordinate diagram of the central axis alignment of the supine and prone 3D surfaces. The abscissa represents sequential pixel coordinates, the ordinate represents height information, and the acupuncture points in the diagram represent intestinal loops. A and B in the diagram correspond to the coordinate information curves of the central axis of the supine and prone 3D surfaces, respectively.

[0054] Figure 5 : is a schematic image showing the alignment of the central axis of the supine three-dimensional curved surface and the prone three-dimensional curved surface. Figure 5 As shown in , L1 and L2 correspond to the central axis of the supine three-dimensional surface and the prone three-dimensional surface, respectively. In some embodiments, a filtering operation can also be performed on the aligned central axis to complete the intestinal rotation (for example Figure 5 Local registration of the part (shown in the middle circle).

[0055] Figure 6 : is an exemplary schematic diagram showing the filtered central axis according to an embodiment of the present application. Figure 6is a schematic diagram of the filtered central axis. In some embodiments, a discrete method, for example, can be used to calculate a coordinate system for a point on the central axis and calculate the rotation angle of two adjacent coordinate systems to determine the rotation amount of the central axis. The central axis is then repeatedly filtered until the rotation amount is less than a target rotation amount (e.g., 25), thereby obtaining the filtered central axis. Based on the filtered central axes of each of the supine and prone three-dimensional surfaces, the distance from the intestinal wall point to the central axis can be calculated to construct a height map corresponding to each of the supine and prone three-dimensional surfaces.

[0056] Figure 7 : is an exemplary schematic diagram showing a height map according to an embodiment of the present application. Figure 7 Shown in the figure is a partial height map corresponding to the supine three-dimensional surface. As mentioned above, by looking for the point corresponding to the closest central axis for all points on the intestinal wall on the supine three-dimensional surface, a distance function from the point to the nearest central axis is constructed. Then, by performing a Laplace filter on the distance function, for example, and applying a Gabor filter to enhance the horizontal and vertical differences in the distance function, the height map is obtained. Similarly, a height map corresponding to the prone three-dimensional surface can be obtained. Then, the respective feature rows are extracted from the height maps corresponding to the supine three-dimensional surface and the prone three-dimensional surface. Specifically, each row of the height map is accumulated to obtain one-dimensional data. , and obtain the feature rows by extracting the local maximum.

[0057] Figure 8 FIG. 1 is an exemplary diagram illustrating the extraction of feature lines according to an embodiment of the present application. Figure 8 The red circle shown in is the local maximum value, which corresponds to the feature line. The horizontal axis represents the serial pixel coordinates, and the vertical axis represents the height information. After obtaining the feature line, the supine three-dimensional surface and the prone three-dimensional surface are extracted based on their respective feature lines to extract the respective intestinal feature points. In an implementation scenario, first, the points whose local minimum values are less than 20% of their surroundings are extracted from the feature line as target points, and the target points whose rotation angles relative to the central axis of the surface meet, for example, a rotation amount of 25 are determined as the supine three-dimensional surface and the prone three-dimensional surface to extract their respective intestinal feature points. Furthermore, intestinal lines can be obtained by connecting the intestinal feature points. The supine three-dimensional surface and the prone three-dimensional surface each correspond to three intestinal lines.

[0058] Figure 9 : is an exemplary schematic diagram showing the intestinal feature points and the corresponding three intestinal lines of the supine three-dimensional curved surface according to an embodiment of the present application. Figure 9The small circles shown in the figure represent intestinal feature points. Intestinal lines can be obtained by connecting the intestinal feature points, including three intestinal lines: the omental band, the mesenteric band, and the independent band (for example, the red, green, and blue lines in the figure). In some embodiments, the present invention also determines the final intestinal lines corresponding to the supine three-dimensional surface and the prone three-dimensional surface by connecting the shortest distances between the respective intestinal feature points and performing smoothing. For example, Figure 10 shown.

[0059] Figure 10 : is a diagram showing the final gut line corresponding to the supine three-dimensional curved surface according to an embodiment of the present application. Figure 10 The final bowel line (i.e., the first bowel line) corresponding to the supine 3D surface is shown. Similarly, the final bowel line (i.e., the second bowel line) corresponding to the prone 3D surface can be obtained. By registering the first and second bowel lines, colorectal surface registration is achieved. In one implementation scenario, the curvature eigenvalue differences between any bowel lines are calculated, and the bowel line with the smallest sum of these differences is selected as the registration result.

[0060] Figure 11 : is an exemplary schematic diagram showing the supine three-dimensional curved surface and the prone three-dimensional curved surface after the gut line is registered according to an embodiment of the present application. Figure 11 Figure (a) shows the supine three-dimensional surface after the intestinal line is aligned. Figure 11 Figure (b) shows the three-dimensional surface in the prone position after intestinal line registration. In some embodiments, a diffeomorphic function can be constructed to optimize the colorectal surface registration results based on the diffeomorphic function. It is understood that a diffeomorphic function is a function that satisfies a one-to-one mapping (i.e., satisfies both injection and surjection) and is continuous and infinitely differentiable.

[0061] In an implementation scenario, first, a gut line (for example, a gut line in a supine 3D surface) can be selected as reference information and recorded as , and record the rest of the gut line information (such as the gut line in the prone three-dimensional surface) as , the above registration result is recorded as In this scenario, the target registration function can be obtained = ,in represents the gradient of the reference information. Then, the registration function is updated based on the target registration function :

[0062] (1)

[0063] in, represents the weight, Indicates reference information. Indicates the rest of the gut line information, In this scenario, by transforming the target registration function Substituting into the above formula (1), we can obtain the updated registration function, which can be expressed as . Further, the initial Beltrami coefficients are calculated based on the updated registration function. In one implementation scenario, the registration function can be linearly interpolated to calculate the initial Beltrami coefficients based on the interpolated registration function. Specifically, the aforementioned initial Beltrami coefficients It can be expressed as follows:

[0064] (2)

[0065] in, , , and , 、 、 Represents the coordinate information of the triangle vertices in the reference information (i.e., the vertices of the surface mesh in the colon surface), 、 、 Represents the coordinate information of the triangle vertices below other intestinal lines, 、 、 The barycentric coefficient of a triangle is expressed as a ratio of the area of the triangle. For example, , Represents the center of gravity. Similarly, we can obtain and Therefore, the aforementioned It is known that by Substituting into the above formula (2), the initial Beltrami coefficient can be obtained.

[0066] Furthermore, a final registration function is obtained based on the initial Beltrami coefficients and the target registration function. In one embodiment, the initial Beltrami coefficients are first corrected so that the final registration function satisfies a diffeomorphism (i.e., a diffeomorphism function is obtained). Then, the target registration function is reconstructed based on the corrected Beltrami coefficients, and the final registration function is determined based on the reconstructed registration function and the target registration function. In one implementation scenario, the initial Beltrami coefficients can be corrected based on the following formula so that the final registration function satisfies a diffeomorphism:

[0067] (3)

[0068] in represents the modified Beltrami coefficient, Indicates the Beltrami coefficient before correction (for example, the initial Beltrami coefficient ). According to the modified Beltrami coefficients, the target registration function can be reconstructed. In an exemplary scenario, assuming that the modified Beltrami coefficients , based on the above formula (1), the following formula can be obtained:

[0069] (4)

[0070] By reorganizing formula (4), we can obtain:

[0071]

[0072] in, , , , , substituting the above parameters into formula (5) and formula (6) we can obtain , from which the target registration function can be reconstructed and the reconstructed registration function can be obtained Then, a final registration function is determined based on the reconstructed registration function and the target registration function, and the final registration function is a diffeomorphic function.

[0073] In one embodiment, in response to the maximum absolute value of the difference between the reconstructed registration function and the target registration function meeting a preset threshold, the reconstructed registration function is used as the final registration function. Specifically, when the maximum absolute value of the difference between the reconstructed registration function and the final registration function is less than the preset threshold, the reconstructed registration function is used as the final registration function. As an example, assuming that the target registration function is denoted as , the reconstructed registration function is recorded as , the preset threshold is recorded as , the reconstructed registration function Registration function with the target The maximum absolute value of the difference between ,but .when < When the reconstructed registration function As the final registration function. On the contrary, when ≥ When using the reconstructed registration function Update the registration function (i.e., use the reconstructed registration function As the target registration function for the next iteration), repeat the above operations, including calculating the Beltrami coefficient based on the updated registration function, correcting the Beltrami coefficient, reconstructing the registration function, and calculating the maximum absolute value of the difference between the current registration function and the previous (or previous iteration) registration function. , until < The target registration function (i.e., differential homeomorphism function) is obtained to further refine the registration result of colorectal surface registration based on the differential homeomorphism function.

[0074] Figure 12 FIG. 1 is an exemplary flow chart showing the overall process for colorectal curved surface registration according to an embodiment of the present application. Figure 12 As shown in , at steps S1101 and S1102, the supine three-dimensional surface and prone three-dimensional surface of the colorectum are collected, respectively, and the supine three-dimensional surface and prone three-dimensional surface are parameterized to corresponding plane domains. Then, at steps S1103 and S1104, the central axis of each of the supine three-dimensional surface and prone three-dimensional surface is extracted, and at step S1105, the central axis is aligned and filtered. Based on the filtered central axis, at steps S1106 and S1107, the height maps corresponding to the supine three-dimensional surface and prone three-dimensional surface are constructed respectively.

[0075] Furthermore, at step S1108 and step S1109, the characteristic lines and intestinal feature points of the supine three-dimensional surface and the prone three-dimensional surface are extracted respectively. After extracting the intestinal feature points, at step S1110 and step S1111, the first intestinal line and the second intestinal line corresponding to the supine three-dimensional surface and the prone three-dimensional surface are obtained based on the intestinal feature points. Then, at step S1112, the first intestinal line and the second intestinal line are aligned to achieve colorectal surface alignment. For more details on colorectal surface alignment, please refer to the above Figure 1 The description of , this application will not be repeated here.

[0076] Figure 13 is an exemplary block diagram illustrating a device 1200 for colorectal curved surface registration according to an embodiment of the present application. It is understood that the device 1200 may include the apparatus of the embodiment of the present application, and the device implementing the solution of the present application may be a single device (e.g., a computing device) or a multifunctional device including various peripheral devices.

[0077] like Figure 13As shown in , the device of the present application may also include a central processing unit ("CPU") 1211, which can be a general-purpose CPU, a dedicated CPU, or other execution unit for information processing and program execution. Furthermore, device 1200 may also include a mass storage device 1212 and a read-only memory ("ROM") 1213. Mass storage device 1212 may be configured to store various types of data, including various supine and prone three-dimensional surfaces related to the colorectum, central axis lines, height maps, bowel lines, algorithm data, intermediate results, and various programs required to operate device 1200. ROM 1213 may be configured to store data and instructions required for power-on self-testing of device 1200, initialization of various functional modules in the system, drivers for the system's basic input / output, and booting the operating system.

[0078] Optionally, device 1200 may also include other hardware platforms or components, such as the illustrated tensor processing unit ("TPU") 1214, graphics processing unit ("GPU") 1215, field programmable gate array ("FPGA") 1216, and machine learning unit ("MLU") 1217. It will be appreciated that while various hardware platforms or components are shown in device 1200, these are merely exemplary and non-limiting, and those skilled in the art may add or remove corresponding hardware as needed. For example, device 1200 may include only a CPU, associated storage devices, and interface devices to implement the method for colorectal surface registration of the present application.

[0079] In some embodiments, to facilitate data transmission and interaction with external networks, the device 1200 of the present application further includes a communication interface 1218, which allows connection to a local area network / wireless local area network ("LAN / WLAN") 1205 via the communication interface 1218, and further to a local server 1206 or the Internet 1207 via the LAN / WLAN. Alternatively or additionally, the device 1200 of the present application may also directly connect to the Internet or a cellular network via the communication interface 1218 using wireless communication technology, such as third generation ("3G"), fourth generation ("4G"), or fifth generation ("5G") wireless communication technology. In some application scenarios, the device 1200 of the present application may also access a server 1208 and a database 1209 on an external network as needed to obtain various known algorithms, data, and modules, and may remotely store various data, such as various data or instructions for presenting, for example, three-dimensional colorectal surfaces in supine and prone positions, central axis lines, height maps, and bowel lines.

[0080] The peripheral devices of the device 1200 may include a display device 1202, an input device 1203, and a data transmission interface 1204. In one embodiment, the display device 1202 may include, for example, one or more speakers and / or one or more visual displays, which are configured to provide voice prompts and / or image video displays for the colorectal surface registration of the present application. The input device 1203 may include other input buttons or controls such as a keyboard, a mouse, a microphone, a gesture capture camera, etc., which are configured to receive audio data input and / or user instructions. The data transmission interface 1204 may include, for example, a serial interface, a parallel interface or a universal serial bus interface ("USB"), a small computer system interface ("SCSI"), a serial ATA, a FireWire ("FireWire"), a PCI Express, and a high-definition multimedia interface ("HDMI"), etc., which are configured for data transmission and interaction with other devices or systems. According to the solution of the present application, the data transmission interface 1204 can receive the supine three-dimensional surface and prone three-dimensional surface of the colorectum collected from the CT device, and transmit the supine three-dimensional surface and prone three-dimensional surface of the colorectum or various other types of data or results to the device 1200.

[0081] The CPU 1211, mass storage 1212, ROM 1213, TPU 1214, GPU 1215, FPGA 1216, MLU 1217, and communication interface 1218 of the device 1200 of the present application can be interconnected via a bus 1219 and can interact with peripheral devices via the bus. In one embodiment, the CPU 1211 can control other hardware components in the device 1200 and its peripheral devices via the bus 1219.

[0082] Combination of the above Figure 13 The apparatus for colorectal curved surface registration that can be used to perform the present application is described. It should be understood that the apparatus structure or architecture herein is merely exemplary, and the implementation and implementation entities of the present application are not limited thereto, but may be modified without departing from the spirit of the present application.

[0083] According to the above description in combination with the accompanying drawings, those skilled in the art can also understand that the embodiments of the present application can also be implemented by software programs. Therefore, the present application also provides a computer-readable storage medium, which stores computer-readable instructions for colorectal surface registration. When the computer-readable instructions are executed by one or more processors, they can be used to implement the present application in combination with the accompanying drawings. Figure 1 、 Figure 12 The described method for colorectal surface registration.

[0084] It should be noted that although the operations of the present method are described in a particular order in the accompanying drawings, this does not require or imply that the operations must be performed in that particular order, or that all of the operations shown must be performed to achieve the desired results. Rather, the steps depicted in the flowcharts may be performed in a different order. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into a single step, and / or a single step may be broken down into multiple steps.

[0085] It should be understood that when the terms "first," "second," "third," and "fourth," etc., are used in the claims, specification, and drawings of this application, they are only used to distinguish different objects, rather than to describe a specific order. The terms "comprise" and "comprising" used in the specification and claims of this application indicate the presence of the described features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or combinations thereof.

[0086] It should also be understood that the terms used in this specification are for the purpose of describing specific embodiments only and are not intended to limit this application. As used in this specification and claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should also be further understood that the term "and / or" as used in this specification and claims refers to any and all possible combinations of one or more of the associated listed items, including and including these combinations.

[0087] Although the implementation methods of this application are as described above, the contents are only examples adopted to facilitate understanding of this application and are not intended to limit the scope and application scenarios of this application. Any technician in the technical field described in this application can make any modifications and changes in the form and details of implementation without departing from the spirit and scope disclosed in this application, but the scope of patent protection of this application shall still be based on the scope defined by the attached claims.

[0088] In addition, the collection and acquisition of various data in this application complies with relevant laws and regulations and is authorized by the data provider. Any organization or individual that needs to obtain external data must obtain authorization in accordance with the law and ensure data security. They must not illegally collect, use, process, or transmit unauthorized or unprotected data, nor illegally buy, sell, provide, or disclose unauthorized or unprotected data.

Claims

1. A method for colorectal surface registration, characterized in that: include: Collecting a supine three-dimensional curved surface and a prone three-dimensional curved surface of the colorectum, and parameterizing the supine three-dimensional curved surface and the prone three-dimensional curved surface to corresponding plane domains respectively; Extracting intestinal feature points of the supine three-dimensional curved surface and the prone three-dimensional curved surface respectively in the corresponding plane domain; Determine a first intestinal line and a second intestinal line corresponding to the supine three-dimensional curved surface and the prone three-dimensional curved surface based on the respective intestinal feature points; The first intestinal line and the second intestinal line are registered to achieve colorectal curved surface registration, The step of extracting the intestinal feature points of the supine three-dimensional curved surface and the prone three-dimensional curved surface in the corresponding plane domain includes: Constructing a height map corresponding to each of the supine three-dimensional curved surface and the prone three-dimensional curved surface in the corresponding plane domain, wherein the height map represents the distance between an intestinal wall point on the supine three-dimensional curved surface or the prone three-dimensional curved surface and the central axis of the curved surface; Accumulating pixel values of each row in the height map corresponding to the supine three-dimensional curved surface and the prone three-dimensional curved surface; Calculate the local maximum value in the accumulated results of the height maps corresponding to the supine three-dimensional curved surface and the prone three-dimensional curved surface; Extracting characteristic lines of the supine three-dimensional curved surface and the prone three-dimensional curved surface based on the points of the local maximum values; Based on the respective feature lines, the intestinal feature points of the supine three-dimensional curved surface and the prone three-dimensional curved surface are extracted.

2. The method according to claim 1, characterized in that Extracting the intestinal feature points of the supine three-dimensional curved surface and the prone three-dimensional curved surface based on the feature line includes: Extracting target points whose local minimum values are less than a preset threshold value from the feature rows; The points of the target points whose rotation angles relative to the central axis of the curved surface satisfy the target central axis rotation amount are determined as the intestinal feature points of the supine three-dimensional curved surface and the prone three-dimensional curved surface, respectively.

3. The method according to claim 1, characterized in that Wherein determining the first intestinal line and the second intestinal line corresponding to the supine three-dimensional curved surface and the prone three-dimensional curved surface based on the respective intestinal feature points includes: The shortest distances between the respective intestinal feature points are connected and smoothed to determine the first intestinal line and the second intestinal line corresponding to the supine three-dimensional curved surface and the prone three-dimensional curved surface.

4. The method according to claim 1 or 3, characterized in that The first intestinal belt lines and the second intestinal belt lines each include three, and registering the first intestinal belt lines and the second intestinal belt lines to achieve colorectal curved surface registration includes: Calculating a target feature value between any one of the first gut lines and any one of the second gut lines; A pair of intestinal lines corresponding to the smallest target eigenvalue is selected for registration to achieve colorectal surface registration.

5. The method according to claim 4, characterized in that The target characteristic value at least includes a curvature characteristic value.

6. The method according to claim 4, characterized in that It also includes: Constructing diffeomorphic functions; The registration result of colorectal surface registration is optimized based on the diffeomorphism function.

7. A device for colorectal curved surface registration, characterized in that: include: processor; A memory having computer instructions for colorectal surface registration stored thereon, wherein when the computer instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

8. A computer-readable storage medium, characterized in that Computer program instructions for colorectal surface registration are stored thereon, and when the computer program instructions are executed by one or more processors, the method according to any one of claims 1-6 is implemented.

Images