Curve registration processing methods, devices, electronic equipment and storage media

Abstract

This invention provides a method, apparatus, electronic device, and storage medium for curve registration. The method for curve registration includes: acquiring M sets of points, each set including a first set of points for identifying the curve to be registered and a second set of points for identifying the template curve; wherein M is an integer greater than or equal to 1; for each set of points in the M sets, determining a set of pose transformation information between the curve to be registered and the template curve; and obtaining target pose transformation information between the curve to be registered and the template curve based on the M sets of pose transformation information.

Description

Technical Field

[0001] This invention relates to the field of medical devices, and more particularly to a method, apparatus, electronic device, and storage medium for curve registration. Background Technology

[0002] Curve registration is frequently required in fields such as machine vision, reverse engineering, automation control, and pattern recognition. Through curve registration, the pose transformation information between two curves can be determined, and based on this pose transformation information, the poses of the two curves can be adjusted to match.

[0003] In existing related technologies, since the two curves to be registered are usually the same shape, it is only necessary to select specific curve points of the curves, such as the start point and the end point. The start point and the end point can form a specific straight line. Then, each curve can correspond to a specific straight line. Then, the pose transformation information between the two specific straight lines corresponding to the two curves can be calculated and used as the pose transformation information between the two curves.

[0004] However, due to limitations in curve extraction methods, the shapes of the two curves that actually need to be registered are often different. If the pose transformation information is marked based on the corresponding specific straight line, it is often impossible to accurately reflect the pose transformation relationship between the two curves. Summary of the Invention

[0005] This invention provides a method, apparatus, electronic device, and storage medium for curve registration, in order to solve the problem of not being able to accurately represent the pose transformation relationship between two curves.

[0006] According to a first aspect of the present invention, a method for curve registration is provided, comprising:

[0007] Obtain M sets of points, each set of points including a first set of points used to identify the curve to be registered, and a second set of points used to identify the template curve; where M is an integer greater than or equal to 1;

[0008] For each of the M sets of points, determine a set of pose transformation information between the curve to be registered and the template curve;

[0009] Based on the pose transformation information of the M groups, the target pose transformation information between the curve to be registered and the template curve is obtained.

[0010] According to a second aspect of the present invention, a curve registration processing apparatus is provided, comprising:

[0011] The acquisition module is used to acquire M sets of points. Each set of points includes a first set of points used to identify the curve to be registered and a second set of points used to identify the template curve; where M is an integer greater than or equal to 1.

[0012] The pose transformation determination module is used to determine a set of pose transformation information between the curve to be registered and the template curve for each of the M sets of points.

[0013] The target pose determination module is used to obtain the target pose transformation information between the curve to be registered and the template curve based on the M sets of pose transformation information.

[0014] According to a third aspect of the present invention, an electronic device is provided, comprising a processor and a memory.

[0015] The memory is used to store code;

[0016] The processor is configured to execute code in the memory to implement the method relating to the first aspect and its alternatives.

[0017] According to a fourth aspect of the invention, a storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the methods involved in the first aspect and its alternatives.

[0018] The curve registration processing method, apparatus, electronic device, and storage medium provided by this invention calculate M sets of pose transformation information based on M sets of points used to identify two curves, and then obtain target pose transformation information based on these M sets of pose transformation information. In this process, each set of pose transformation information reflects the transformation information of the two curves in that set of points, and the target pose transformation information calculated by combining the M sets of transformation information can comprehensively and fully reflect the pose transformation relationships reflected by different sets of points.

[0019] In a further optional scheme, the curve points sampled from the curve to be registered and / or the template curve can be interpolated, thereby effectively increasing the number of curve points of the corresponding curve (i.e., the curve to be registered and / or the template curve), thus providing a more sufficient basis for curve registration and helping to improve the accuracy of curve registration.

[0020] Based on this, since the template curve and the registration curve are usually not exactly the same, if the interpolation scale is too fine (i.e., the value or range of the distance between adjacent curve points after sampling and interpolation is too small), although it can accurately represent the shape of the curve details, it will also accurately represent the differences between the curves. Consequently, this accurate representation of the differences will affect the accuracy of the registration between the curves. If the interpolation scale is too coarse (i.e., the value or range of the distance between adjacent curve points after sampling and interpolation is too large), it will result in a small number of sampling points, which will also reduce the accuracy of the registration between the curves.

[0021] In response to this, in an optional embodiment of the present invention, interpolation is performed with different scale requirements, and based on the first point set and the second point set formed after each interpolation, a set of pose transformation information between curves is determined, and then M sets of pose transformation information are integrated. It can be seen that the integrated target pose transformation information can take into account the curve registration results under multiple scales, which helps to compensate for the influence of different scales on the registration accuracy and further improve the accuracy of registration between curves. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a schematic diagram illustrating an application scenario in an exemplary embodiment of the present invention;

[0024] Figure 2 This is a schematic flowchart of a curve registration processing method in an exemplary embodiment of the present invention;

[0025] Figure 3 This is a schematic diagram of the process of obtaining the first point set and the second point set in an exemplary embodiment of the present invention;

[0026] Figure 4 This is a comparative schematic diagram of two interpolation methods in an exemplary embodiment of the present invention;

[0027] Figure 5 This is a schematic flowchart illustrating the positioning pose transformation information in an exemplary embodiment of the present invention;

[0028] Figure 6 This is a schematic flowchart illustrating the process of determining target alignment relationships in an exemplary embodiment of the present invention;

[0029] Figure 7 This is a schematic diagram of the process for determining the registration coordinate system and the template coordinate system in an exemplary embodiment of the present invention;

[0030] Figure 8 This is a schematic diagram of the calculation principle in an exemplary embodiment of the present invention;

[0031] Figure 9 This is a schematic flowchart illustrating the process of determining the orientation of the three coordinate axes of the registration coordinate system in an exemplary embodiment of the present invention.

[0032] Figure 10a This is a schematic diagram of the registration result in the prior art. Figure 1 ;

[0033] Figure 10b This is a schematic diagram of the registration result in the prior art. Figure 2 ;

[0034] Figure 10c This is a schematic diagram of the registration result in a specific example of the present invention;

[0035] Figure 11 This is a flowchart illustrating the process of determining the orientation of the three coordinate axes of the template coordinate system in an exemplary embodiment of the present invention.

[0036] Figure 12 This is a schematic diagram of the process for determining the target pose transformation matrix in an exemplary embodiment of the present invention;

[0037] Figure 13 This is a schematic diagram of the process for determining the final pose transformation matrix in an exemplary embodiment of the present invention;

[0038] Figure 14 This is a schematic diagram of the program modules of a curve registration processing device in an exemplary embodiment of the present invention;

[0039] Figure 15 This is a schematic diagram of the program modules of the curve registration processing device in another exemplary embodiment of the present invention;

[0040] Figure 16 This is a schematic diagram of the structure of an electronic device in an exemplary embodiment of the present invention. Detailed Implementation

[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0042] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0043] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0044] Figure 1 This is a schematic diagram illustrating an application scenario in an exemplary embodiment of the present invention. For example... Figure 1 As shown, in scenarios requiring curve registration, an acquisition unit 101 can be used. The acquisition unit 101 can be understood as any device or combination of devices that directly or indirectly acquires the curve to be registered. The acquisition unit 101 can communicate directly or indirectly with the data processing unit 102. Based on the information acquired by the acquisition unit 101, the data processing unit 102 can realize the curve registration process. The data processing unit 102 can be any device or combination of devices with data processing capabilities.

[0045] The curves to be registered include the curve to be registered and the template curve, which can be three-dimensional or two-dimensional.

[0046] In mathematics, for any spatial curve, the function Curvature(x,y,z) can be defined as a functional representation of the curve. In practice, curves are often described using discrete sampled data, for example, as follows:

[0047] Template curve {α i |i=1,...,N α};

[0048] The curve to be registered {β i |i=1,...,N β}

[0049] α i This refers to the i-th curve point among the curve points of the template curve (specifically, the curve points before interpolation);

[0050] N α This indicates the number of curve points in the template curve (specifically, the curve points before interpolation);

[0051] β i This refers to the i-th curve point among the curve points of the curve to be registered (specifically, the curve points before interpolation);

[0052] N β This indicates the number of curve points in the curve to be registered (specifically, the curve points before interpolation).

[0053] As can be seen, the number of points between the template curve and the curve to be registered may be inconsistent. A certain part of the template curve may have the same part as the curve to be registered. That is to say, there is a sub-curve of a certain length on the template curve that is the same as the curve to be registered. The goal of registration is to find out the rigid body transformation relationship of this common part in the two curves, that is, the rotation matrix and translation matrix in three-dimensional space (which can be understood as a kind of pose transformation information, target pose transformation information).

[0054] In one embodiment, the registration curve characterizes the shape of the bronchoscope used to insert into the bronchial tree; furthermore, the acquisition unit 101 may be, for example, a shape sensor for detecting the shape of the bronchoscope, and the template curve characterizes the shape of a tracheal segment in the bronchial tree.

[0055] In one embodiment, the template curve and its corresponding data (e.g., the second set of points representing the template curve, and the second feature points selected from the second set of points) can be preset and do not need to be redefined for each curve to be registered. For example, multiple curve points can be pre-determined through sampling, interpolation, etc., so that for the curve to be registered, M points can be directly selected from the multiple sets of second points to form a second set, or N2 points can be directly selected from them as second feature points, etc. Of course, the data corresponding to the template curve can also be redefined for each curve to be registered, and this embodiment does not limit this.

[0056] In the embodiments of this invention, please refer to Figure 2 The methods for curve registration include:

[0057] S201: Obtain the M set of points;

[0058] Each set of points includes a first set of points used to identify the curve to be registered, and a second set of points used to identify the template curve;

[0059] It should be noted that the first point set (i.e., the M first point sets) in the M point sets all identify the same curve to be registered; the second point set (i.e., the M second point sets) in the M point sets all identify the same target curve. The number of first curve points included in each first point set and the number of second curve points included in each second point set can be the same or different; this embodiment does not impose any limitations.

[0060] Where M is an integer greater than or equal to 1; in a specific example, M can be an integer greater than or equal to 2; in one example, the M set of points can be a set of points obtained by sampling with M different scale requirements; in another example, the M set of points can be a set of points obtained by interpolation with M different scale requirements after sampling; in yet another example, the M set of points can also be a set of points of different curve segments of a curve.

[0061] As can be seen, the first point set may include curve points obtained by sampling the curve to be registered, and in some examples, it may also include curve points obtained by interpolating the curve points; the second point set may include curve points obtained by sampling the template curve, and in some examples, it may also include curve points obtained by interpolating the curve points.

[0062] In one example, the first and second point sets in each set are obtained using the same scale requirement. For instance, M = 2, including point set 1 and point set 2. Point set 1 includes the first and second point sets, obtained by sampling and / or interpolation using scale 1; point set 2 includes the first and second point sets, obtained by sampling and / or interpolation using scale 1.

[0063] S202: For each group in the M sets of points, determine a set of pose transformation information between the curve to be registered and the template curve;

[0064] The pose transformation information can be understood as representing a pose transformation relationship between the curve to be registered and the template curve. Through this pose transformation relationship, the poses of the curve to be registered and the template curve can be mutually converted. Furthermore, different sets of points may reflect different shapes and positional characteristics of the curve. In this case, the corresponding M sets of pose transformation information can reflect the pose transformation information obtained when different shapes and positional characteristics are satisfied, thus providing a sufficient, comprehensive and reliable basis for determining the target pose transformation information.

[0065] In one embodiment, the pose transformation information may be the pose transformation matrix between the curve to be registered and the template curve.

[0066] S203: Based on the pose transformation information of the M group, obtain the target pose transformation information between the curve to be registered and the template curve.

[0067] The target pose transformation information can be understood as representing a pose transformation relationship between the curve to be registered and the template curve. Through this pose transformation relationship, the poses of the curve to be registered and the template curve can be adjusted to match.

[0068] In one embodiment, the target pose transformation information can be selected from one set of M sets of pose transformation information; or it can be obtained by combining the M sets of pose transformation information.

[0069] In the above process, since the target pose transformation information is based on M sets of pose transformation information, the target pose transformation information calculated by the target pose transformation information can take into account and fully reflect the pose transformation relationship reflected by different sets of points. Furthermore, since the determined target pose transformation information is no longer limited to the pose transformation relationship reflected by a single set of points, the final determined target pose transformation information can accurately reflect the pose transformation relationship of the two curves.

[0070] In one implementation method, please refer to Figure 3 Obtain M sets of points, including:

[0071] S301: Sample the curve points of the curve to be registered and the curve points of the template curve;

[0072] In a specific example, the curve to be registered and the template curve can be sampled using the same or different sampling rates. In one embodiment, if the curve points of the two curves are difficult to align due to different sampling rates, interpolation of the sampling points is required, as detailed in step S302.

[0073] In some examples, the sampled curve points can be filtered. Common filtering methods include high-pass filtering, low-pass filtering, Gaussian filtering, and mean filtering.

[0074] S302: According to different scale requirements, interpolate the curve points in the curve to be registered and / or the template curve M times to obtain new curve points of the corresponding curve, and after each interpolation, form a first set of curve points of the curve to be registered and a second set of curve points of the template curve.

[0075] The scale requirement characterizes the required value or range of distances between adjacent curve points in the first and second point sets. Furthermore, a higher scale requirement results in a higher degree of dispersion in the interpolated point set; conversely, a lower scale requirement results in a lower degree of dispersion in the interpolated point set.

[0076] In a specific example, the interpolation process requiring different scales can be processed in parallel using multi-threading, achieving parallelization of multi-scale computation and improving computational efficiency. For instance, a server (one or more) can enable multiple threads to perform interpolation at different scales, obtaining a point set corresponding to each scale. Optionally, these multiple threads can further process the interpolated point sets individually, such as determining pose transformation information for the set of points. Thus, each thread can obtain a set of points and process that set, achieving parallel processing of multiple point sets and improving efficiency.

[0077] In one example, linear interpolation can be used when interpolating for different scale requirements; however, since the curve points are in space, i.e. the curve is three-dimensional, using linear interpolation may result in sharp, acute angles in the curve, which would make the interpolated curve points unable to accurately describe the curve, causing distortion.

[0078] In another example, spherical interpolation can be used for interpolation. The method of spherical interpolation can be found in relevant technologies, which will not be elaborated here.

[0079] See Figure 4 Curves were obtained using linear interpolation and spherical interpolation, respectively. Linear interpolation can be illustrated as shown in Figure 401, while spherical interpolation can be illustrated as shown in Figure 402. It is evident that spherical interpolation yields a relatively smooth transition.

[0080] The set of curve points in the interpolated template curve can be represented as:

[0081]

[0082] The set of curve points in the interpolated curve to be registered can be characterized as:

[0083]

[0084] This represents the i-th curve point among the curve points of the interpolated template curve;

[0085] This represents the i-th curve point among the curve points of the curve to be registered after interpolation;

[0086] N stepα This indicates the number of curve points in the curve to be registered after interpolation;

[0087] N stepβ This indicates the number of curve points in the template curve after interpolation.

[0088] It should be noted that the curve points of the template curve formed after interpolation may or may not include the sampled curve points, depending on the interpolation scale and sampling interval.

[0089] In the above scheme, interpolation is performed on the curve points sampled from the curve to be registered and / or the template curve. This effectively increases the number of curve points of the corresponding curve (i.e., the curve to be registered and / or the template curve), providing a more sufficient basis for curve registration and helping to improve the accuracy of curve registration. On the other hand, it unifies the interval between the curve points in the first and second point sets of each point set, making it easier to align the first and second point sets and improve processing efficiency.

[0090] Based on this, since the template curve and the registration curve are usually not exactly the same, if the interpolation scale is too fine (i.e., the value or range of the distance between adjacent curve points after sampling and interpolation is too small), although it can accurately represent the shape of the curve details, it will also accurately represent the differences between the curves. Consequently, this accurate representation of the differences will affect the accuracy of the registration between the curves. If the interpolation scale is too coarse (i.e., the value or range of the distance between adjacent curve points after sampling and interpolation is too large), it will result in a small number of sampling points, which will also reduce the accuracy of the registration between the curves.

[0091] The integrated target pose transformation information can comprehensively reflect curve registration results at multiple scales, helping to compensate for the impact of different scales on registration accuracy and further improving the accuracy of inter-curve registration. For example, at a larger scale, the feature points used for registration can effectively reflect the large-scale change trend of the curve, but the shape at the details is not accurate. Therefore, registration based on this larger scale can accurately reflect the relative pose relationship between the two curves in terms of the large-scale change trend (e.g., the overall change trend), but the details may not be accurate. Conversely, at a smaller scale, the shape at the details of the curve can be effectively reflected, but this often leads to a more concentrated distribution of the feature points actually used for registration between the curves. This is because at a smaller scale, there are more curve points, and the undulations of the curve are more significant, which will lead to the selection of... Feature points are concentrated on certain curve segments with significant fluctuations (e.g., when selecting feature points based on torsion and / or curvature, the selected feature points will be concentrated on curve segments with greater curvature). In this case, the registration result can accurately adapt to the pose transformation relationship of such curve segments of the two curves, but it may not be able to accurately adapt to the relative pose relationship of the two curves in a large range of change trends (e.g., overall change trend). Therefore, the integrated target pose transformation information can take into account the advantages of curve registration results at different scales, thereby taking into account both the pose transformation relationship at details and the pose transformation relationship in a large range, effectively improving accuracy.

[0092] In one implementation method, please refer to Figure 5 For each of the M sets of points, a set of pose transformation information between the curve to be registered and the template curve is determined based on the first and second point sets in that set, including:

[0093] S501: Select N1 curve points from the first point set as N1 first feature points, and select N2 curve points from the second point set as N2 second feature points;

[0094] N1 and N2 are integers greater than or equal to 2;

[0095] S502: Based on the N1 first feature points and N2 second feature points, determine the target alignment relationship between the first point set and the second point set in this set of points.

[0096] The target alignment relationship represents that: A first curve points in the first point set and A second curve points in the second point set are mutually matched;

[0097] S503: The pose transformation information is determined by using the first curve point and the second curve point that match each other.

[0098] The first feature point can be understood as any curve point that can describe the curve shape and / or position characteristics of the first point set; the second feature point can be understood as any curve point that can describe the curve shape and / or position characteristics of the second point set. In one example, the first and second feature points are usually curve points with relatively obvious features. Any method of extracting feature points in this field can be applied here as an optional solution.

[0099] In the differential geometry of curves, the curvature and torsion of a curve are rotation- and translation-invariant. Therefore, in one embodiment, curvature and / or torsion can be used as feature information to extract first feature points and second feature points, for example, curve points with larger feature information. Specifically, step S501 may include:

[0100] First feature information is extracted from each first curve point in the first point set, and N1 first feature points are selected from the first point set based on the first feature information. The first feature information characterizes the curvature and / or torsion of the curve to be registered at the corresponding first curve point. In one example, the N1 first curve points with the highest first feature information values ​​can be selected from the first point set as the N1 first feature points. In another example, the curve can be divided into multiple curve segments, and multiple first curve points with the highest first feature information in each curve segment can be selected as multiple first feature points.

[0101] Extract the second feature information of each second curve point in the second point set, and select the N2 second feature points in the second point set based on the second feature information; the second feature information characterizes the curvature and / or torsion of the template curve at the corresponding second curve point; in one example, the N2 second curve points with the highest second feature information values ​​can be selected from the second point set as the N2 second feature points; in another example, the curve can be divided into multiple curve segments, and then multiple second curve points with the highest second feature information in each curve segment can be selected as multiple second feature points;

[0102] N1 and N2 can be equal, for example, both being 10 or 15; or they can be unequal, for example, N1 = 10 and N2 = 15.

[0103] In one example, in addition to selecting features from high to low, further selection can be made based on the interval between curve points. For example, the interval between the selected feature points may be required to meet preset requirements, such as requiring that the feature points are not adjacent curve points; or requiring that the feature points be spaced apart by a specified number of curve points. For example, in the first point set... In the selection process, the selection is first made according to the feature information from highest to lowest. If the point with the highest feature information is selected first... As a feature point, select the point with the second highest feature information. Because of this and If the requirement that "feature points are not adjacent curve points" is not met, then it will not be... Set them as feature points, and then continue to select from high to low feature information until the number of selected feature points meets the condition.

[0104] In one example, the first feature information may be determined based on curvature and torsion, such as a weighted sum or average of curvature and torsion; or curvature may be used alone as the first feature information, or torsion may be used alone as the first feature information. The method for determining the second feature information corresponds to the first feature information, for example, it may be determined based on curvature and torsion, or curvature or torsion may be used alone as the first feature information.

[0105] The target alignment relationship represents the mutual matching of A first curve points in the first point set with A second curve points in the second point set. It should be noted that this mutual matching relationship can mean that all first curve points in the first point set match with curve points in the second point set, i.e., A equals the number of curve points in the first point set; or it can mean that some first curve points in the first point set match with all or part of the second curve points in the second point set, i.e., A is not greater than the number of curve points in the first point set.

[0106] In one example of step S503, the pose transformation relationship between the registration coordinate system based on the curve to be registered and the template coordinate system based on the template curve can be calibrated using the matching first curve points and second curve points, and the pose transformation relationship can be used as the pose transformation information. Thus, a pose transformation information can be determined for each of the M sets of points, resulting in M ​​sets of pose transformation information.

[0107] In the above scheme, multiple curve points are selected as feature points through feature extraction to determine the alignment relationship. Using curve points with higher feature values ​​ensures the accuracy of the alignment while avoiding the computational complexity of traversing and aligning all curve points, thus improving processing efficiency. Furthermore, based on the mutually matching first and second curve points obtained through alignment, pose transformation information is calibrated, effectively ensuring the accuracy of the pose transformation information.

[0108] In one implementation method, please refer to Figure 6 Based on the N1 first feature points and N2 second feature points, the target alignment relationship between the first point set and the second point set in this set of points is determined, including:

[0109] S601: Based on the N1 first feature points and N2 second feature points, determine multiple candidate alignment relationships between the first point set and the second point set in this set of points;

[0110] The candidate alignment relationship represents the mutual matching of several first curve points in the first point set with several second curve points in the second point set; in one example, the candidate alignment relationship may be randomly determined; in another example, a matching second feature point may be searched for for each first feature point based on a preset search logic.

[0111] In one example, step S601 may be to take the continuously matched curve points in the first point set and the second point set as a set of candidate alignment relationships. For example, preliminary matching may be performed first based on the first feature point and the second feature, and then it may be determined whether the curve points near the first feature point match the curve points near the second feature point.

[0112] In one example, step S601 may include:

[0113] For each of the N1 first feature points, a second feature point that matches the first feature point is determined from the N2 second feature points to determine the mutually matching first feature point and second feature point; the feature error of the mutually matching first feature point and second feature point is less than a preset threshold.

[0114] in:

[0115] After each pair of matching first and second feature points is determined (e.g., the m-th second feature point α), m With the nth first feature point β n Once a pair of features is determined to be a match (where m may or may not be equal to n), the first feature point (e.g., the first feature point β) in the matched feature point pair is used. n ) as the first reference point, the second feature point (e.g., the second feature point α) m) as the second reference point; then determine the adjacent first curve points (e.g., first curve point β) of the first reference point in the first set. n+1 β n-1 ), and the adjacent second curve points (e.g., second curve point α) of the second reference point in the second set. m+1 α m-1 Are they mutually matching feature point pairs?

[0116] If the adjacent first curve point and the adjacent second curve point match each other, then the adjacent first curve point is taken as the first reference point and the second curve point is taken as the second reference point, and the process returns to determine whether the adjacent first curve points of the first reference point in the first set and the adjacent second curve feature points of the second reference point in the second set match each other.

[0117] The following is a specific example:

[0118] In one embodiment, matching second feature points can be determined sequentially for each first feature point along a specified direction, for example, β1, β2, β3, etc. Simultaneously, when determining matching second feature points for a specified first feature point, the matching of the second feature points can be determined sequentially along the specified direction. For example, for the first feature point β1, matching can be performed in the order of α1, α2, α3, etc., until the first matching feature point (e.g., α2) is found, or until all second feature points are determined to be mismatched; then for the first feature point β2, matching can be performed again in the order of α1, α2, α3, etc., and so on, until all matching of the first feature points is completed.

[0119] In this process, after determining a matching second feature point for any first feature point, it can also be determined whether the adjacent first curve points of the first feature point in the first set match the adjacent second curve points of the second feature point in the second set. Taking β4 as the first feature point and α6 as the second feature point in the second set as an example, it can be determined that the two feature points match.

[0120] Then, using β4 as the first reference point and α6 as the second reference point, we determine whether adjacent points in the first and second point sets match along two directions. For example, we first match along the direction of the endpoint of the curve (including the curve to be registered and the template curve). For instance, if the first curve points in the first set are arranged in the order of the curve to be registered from the start to the end, and the second curve points in the second set are arranged in the order of the template curve from the start to the end, we can determine whether adjacent β5 (adjacent to the first reference point β4) matches α7 (adjacent to the second reference point α6).

[0121] If a match is found, β5 is taken as the first reference point and α7 as the second reference point. The process continues to determine whether adjacent curve points in the two point sets match... This continues until the adjacent first feature points of the first reference point and the adjacent second feature points of the second reference point do not match, or the last point of the first or second point set is reached.

[0122] Next, using the same method, matching is performed again in the direction of the curve starting point, that is, determining whether the adjacent β3 (adjacent to the first reference point β4) and α5 (adjacent to the second reference point α6) match;

[0123] If a match is found, β3 is taken as the first reference point, α5 as the second reference point, and so on, until the adjacent first feature points of the first reference point and the adjacent second feature points of the second reference point do not match, or the first point of the first point set or the second point set is reached.

[0124] This completes one matching process, and the matching curve points determined along the two directions can be used as a set of candidate alignment relationships.

[0125] It should be noted that the above-described matching method, which first proceeds along the direction "towards the end of the curve" and then along the direction "towards the start of the curve", is merely an illustrative example. In practical applications, the matching can be performed simultaneously along both directions, or first along the direction "towards the start of the curve" and then along the direction "towards the end of the curve". This embodiment does not impose any limitations on this method.

[0126] It should be noted that this embodiment does not limit the arrangement order of the curve points in the first and second point sets. The arrangement can be in the order of the curve points from the start point to the end point in the curve to be registered or the template curve, or it can be in the order of the end point to the start point.

[0127] In one example, the start and end points of the curve can be preset. For instance, if the curve to be registered is the shape of a bronchoscope and the template curve is the shape of a tracheal segment, then the laryngeal inlet can be set as the start point.

[0128] In another example, the start and end points of the curve can be determined based on the characteristic information of the curve points. For example, the endpoint closer to the curve point with the highest characteristic information can be used as the start point; or as the end point. Taking the two endpoints of the curve to be registered as curve points O and P as an example, first determine the curve point Q with the highest characteristic information in OP. Then, based on the ratio of OQ to OP, if it is less than 50%, set O as the start point and P as the end point; if it is greater than 50%, set P as the start point and Q as the end point. At the same time, the same method is used to judge the template curve. Alternatively, in one example, if the ratio of OQ to OP is equal to 50%, the endpoint closer to the curve point with the second highest characteristic information can be used as the start point.

[0129] Through the above process, traversal of feature points can be achieved. Based on this:

[0130] In one example, if the adjacent first feature point (e.g., first feature point β) n+1 ) and the adjacent second feature point (e.g., second feature point α) m+1 If there is no match, a candidate alignment relationship is determined based on all the already determined mutually matching feature point pairs. This can be understood as: when a mismatch is found, the traversal of feature points is stopped, and after stopping the traversal, all the already determined mutually matching feature point pairs are determined as a candidate alignment relationship.

[0131] In another example, if the adjacent first feature point (e.g., first feature point β) n+1 ) and the adjacent second feature point (e.g., second feature point α) m+1 If there is a mismatch, then these two feature points are not considered a matching feature point pair. However, the adjacent first feature point and the adjacent second feature point can be used as the first reference point and the second reference point to determine new adjacent first feature points (e.g., first feature point β). n+2 ) and adjacent second feature points (e.g., second feature point α) m+2 In this example, the traversal of feature points can be stopped only after all the first feature points have been traversed or the number of first feature points traversed has reached a preset value. After stopping the traversal, all the mutually matching feature point pairs that have been determined are identified as a candidate alignment relationship.

[0132] Please refer to Figure 6 Based on the N1 first feature points and N2 second feature points, determining the target alignment relationship between the first point set and the second point set in this set of points may further include:

[0133] S602: For each candidate alignment relationship, calculate the feature error between each pair of matching first curve points and second curve points, and determine the feature error evaluation information under the candidate alignment relationship based on the feature error between each pair of matching first curve points and second curve points.

[0134] The feature error is determined based on the first feature information of the first curve point and the second feature information of the second curve point; correspondingly, the feature error evaluation information can be understood as any information that evaluates the feature error of all (or some) curve points under the candidate alignment relationship as a whole, which can be the statistical value of the feature error (e.g., summation, average, weighted summation, etc., at least one of them).

[0135] S603: Based on the feature error evaluation information of each of the multiple candidate alignment relationships, select one candidate alignment relationship as the target alignment relationship from the multiple candidate alignment relationships.

[0136] In one example of step S603, it may include: selecting the candidate alignment relationship with the smallest feature error evaluation information value from the multiple candidate alignment relationships as the target alignment relationship;

[0137] In another example of step S603, it may include: selecting several candidate alignment relationships from the multiple candidate alignment relationships whose feature error evaluation information satisfies preset conditions, and taking the candidate alignment relationship with the most mutually matching curve point pairs as the target alignment relationship; wherein, the mutually matching curve point pairs include a first feature point and a second feature point that match each other.

[0138] The preset conditions could be, for example, selecting one or more feature error evaluation information values ​​that fall within a preset range from a variety of candidate alignment relationships; or, for example, selecting one or more feature error evaluation information values ​​that have the smallest statistical values ​​from a variety of candidate alignment relationships.

[0139] In a specific example of steps S601 to S603 above, the implementation process can be as follows:

[0140] The curvature and torsion of the curve are calculated and saved as the curve's feature information (i.e., the first feature information and the second feature information). At the same time, the curve points in the first point set and the curve points in the second point set are sorted according to the magnitude of the feature information in the curve. The feature points with larger feature information are selected to obtain N1 first feature points and N2 second feature points.

[0141] The set of the second characteristic points of the template curve can be represented as

[0142] The set of the first characteristic points of the curve to be registered can be represented as:

[0143] in:

[0144] Let i represent the i-th second feature point among N1 second feature points;

[0145] This represents the i-th first feature point among N2 first feature points;

[0146] Then, the process can be repeated sequentially according to step S601 to determine several sets of candidate alignment relationships. Taking a candidate alignment relationship that includes A pairs of mutually matching curve points as an example, the feature errors of each pair of curve points can be determined, and then the feature error evaluation information corresponding to the template curve and the curve to be registered can be determined. For example, the sum of the feature errors of the pair of curve points can be used as the feature error evaluation information. For example, the first mutually matching curve point β can be determined.k (k = 1, 2, ..., A) and the second curve point α k The characteristic error ε between k From this, we can obtain ε1, ε2, ..., ε A Then we can calculate ε1, ε2, ..., ε A The sum is used as information for feature error evaluation.

[0147] Based on the feature error evaluation information of each candidate alignment relationship, the candidate alignment relationship with the smallest feature error evaluation information, or the candidate alignment relationship with the most curve point pairs that meet preset conditions, can be selected as the target alignment relationship. For example:

[0148] The set of the second curve points of the template curve under the target alignment relationship can be represented as:

[0149] The set of the first curve points of the curves to be registered under the target alignment relationship can be represented as:

[0150] This represents the second curve point after alignment (i.e., under the target alignment relationship);

[0151] This represents the first curve point after alignment (i.e., under the target alignment relationship);

[0152] N align This indicates the number of the first feature points after alignment (i.e., under the target alignment relationship) (i.e., the same as the number A mentioned above), which is also the number of the second feature points after alignment (i.e., under the target alignment relationship).

[0153] After determining the target alignment relationship between the first feature point and the second feature point, the pose transformation information can be calibrated based on the A first curve points and the corresponding A second curve points represented by the target alignment relationship.

[0154] Figure 7 This is a schematic diagram illustrating the process of determining the registration coordinate system and the template coordinate system in an exemplary embodiment of the present invention. Figure 7 As shown, the method for determining the registration coordinate system includes steps S701-S704, and the method for determining the template coordinate system includes steps S705-S708. These will be described in detail below:

[0155] Please refer to Figure 7 The registration coordinate system is determined through the following process:

[0156] S701: Obtain the set of first curve points formed by the A first curve points;

[0157] S702: Calculate the first covariance matrix of the first set of curve points;

[0158] S703: Determine the eigenvalues ​​of the first covariance matrix, and based on the eigenvalues ​​of the first covariance matrix, determine the eigenvectors of the first covariance matrix;

[0159] S704: Determine the registration coordinate system based on the eigenvectors of the first covariance matrix;

[0160] Please refer to Figure 7 The template coordinate system is determined through the following process:

[0161] S705: Obtain the set of second curve points formed by the A second curve points;

[0162] S706: Calculate the second covariance matrix of the second set of curve points;

[0163] S707: Determine the eigenvalues ​​of the second covariance matrix, and based on the eigenvalues ​​of the second covariance matrix, determine the eigenvectors of the second covariance matrix;

[0164] S708: Determine the template coordinate system based on the eigenvectors of the second covariance matrix.

[0165] Based on the above steps, the registration coordinate system and the template coordinate system can be determined. Based on this, the pose transformation information between the curve to be registered and the template curve can be further determined, as described in step S709.

[0166] In one example, step S709 and Figure 5 The process in step S503 shown, which involves “using the first and second curve points that match each other to calibrate the pose transformation relationship between the registration coordinate system based on the curve to be registered and the template coordinate system based on the template curve, and using the pose transformation relationship as the pose transformation information,” is the same or similar. For the same or similar content, it will not be described again here.

[0167] The geometric meaning of the eigenvectors mentioned above can be understood as follows: transformations of the covariance matrix on the eigenvectors only scale the vectors, while keeping the direction of the eigenvectors unchanged. Eigenvalues ​​represent the importance of that eigenvalue.

[0168] The following is combined with Figure 8 Describe the methods for "determining eigenvalues ​​and eigenvectors".

[0169] First, we can target the set of second curve points of the template curve. The set of the first curve points of the curve to be registered Calculate the center point of each discrete point data point. For example, if the curve points mentioned above are represented by coordinate values ​​in a preset coordinate system (such as the CT coordinate system, the coordinate system of a sensor used to detect the shape of an endoscope, etc.), then the center point can be the average value of each coordinate value in the set along each coordinate axis. The center point of the set of second curve points is denoted as <α>. j >;The center point of the set of points of the first curve is denoted as <β j Then, based on the center point of the set of curve points, the curve points in the set are centered. Taking the second set of curve points as an example, each curve point in the second set is subtracted from the center point <α. j This yields the second set of curve points after centering. A similar process is then performed on the first set of curve points to obtain the first set of curve points after centering.

[0170] For the centered set (which can be represented in matrix form), the corresponding covariance matrix can be calculated. For example, the second covariance matrix is ​​calculated for the centered second set of curve points, and the first covariance matrix is ​​calculated for the centered first set of curve points.

[0171] Subsequently, the PCA (Principal Component Analysis) algorithm is used to calculate several eigenvalues ​​corresponding to the two covariance matrices, and the corresponding eigenvector is determined based on each eigenvalue. It should be noted that this example uses the PCA algorithm; in practical applications, any other method in this field can be used to calculate the eigenvalues ​​and eigenvectors of the covariance matrix.

[0172] Taking several first eigenvectors corresponding to the first covariance matrix as an example, these first eigenvectors are normalized by ordering their corresponding eigenvalues ​​from largest to smallest, resulting in normalized vectors. For a curve in three-dimensional space, the eigenvectors corresponding to the two largest and second largest eigenvalues ​​are taken and denoted as follows: Because of this Since they are perpendicular to each other, performing a vector product operation on these two eigenvectors will yield a result perpendicular to each other. The third unit vector, denoted as

[0173] Using the same method, for the second covariance matrix corresponding to the second set of curve points, we can also obtain three mutually perpendicular unit vectors, denoted as...

[0174] In one example, the eigenvectors of the first covariance matrix include a first eigenvector and a second eigenvector; the first eigenvector corresponds to the largest eigenvalue of the first covariance matrix (e.g., the one described above). (corresponding to), the second eigenvector corresponds to another eigenvalue of the first covariance matrix after the largest eigenvalue (e.g., the one mentioned above). correspond);

[0175] Similarly, the eigenvectors of the second covariance matrix can also include the first and second eigenvectors corresponding to the largest and second largest eigenvalues.

[0176] In one embodiment, for the curve to be registered, the registration coordinate system can be determined based on the eigenvectors of the first covariance matrix. For example, as described above. It can be determined that the three coordinate axes of the registered coordinate system are parallel. The following section combines... Figure 9 This section introduces methods for determining the direction of coordinate axes.

[0177] Please refer to Figure 9 The process of determining the registration coordinate system based on the eigenvectors of the first covariance matrix may include, for example:

[0178] S901: Obtain the vector from the first first curve point to the last first curve point in the first set of curve points as the first vector;

[0179] S902: By comparing the first vector with the first eigenvector of the first covariance matrix, the direction of the first eigenvector of the first covariance matrix or the inverse vector of the first eigenvector is selected as the positive direction of the first coordinate axis of the registration coordinate system.

[0180] In a specific example, step S902 may include:

[0181] Perform a dot product operation between the first vector and the first eigenvector of the first covariance matrix to obtain the first dot product;

[0182] If the first dot product is greater than zero, then the direction of the first eigenvector of the first covariance matrix is ​​determined as the positive direction of the first coordinate axis of the registration coordinate system;

[0183] If the first dot product is not greater than zero, then the direction of the inverse vector of the first eigenvector of the first covariance matrix is ​​determined as the positive direction of the first coordinate axis of the registration coordinate system;

[0184] S903: Calculate the midpoint of the line connecting the first first curve point and the last first curve point in the first set of curve points as the first midpoint, calculate the first curve point in the middle position in the first set of curve points as the first midpoint, and obtain the second vector from the first midpoint to the first midpoint.

[0185] S904: By comparing the second vector with the second eigenvector of the first covariance matrix, the direction of the second eigenvector of the first covariance matrix or the inverse vector of the second eigenvector of the first covariance matrix is ​​selected as the positive direction of the second coordinate axis of the registration coordinate system.

[0186] In a specific example, step S904 may include:

[0187] Perform a dot product operation between the second vector and the second eigenvector of the first covariance matrix to obtain the second dot product;

[0188] If the second dot product is greater than zero, then the direction of the second eigenvector of the first covariance matrix is ​​determined as the positive direction of the second coordinate axis of the registration coordinate system;

[0189] If the second dot product is not greater than zero, then the direction of the inverse vector of the second eigenvector of the first covariance matrix is ​​determined as the positive direction of the second coordinate axis of the registration coordinate system;

[0190] S905: Perform a vector product operation on the positive directions of the first coordinate axis and the second coordinate axis of the registration coordinate system to obtain a first vector product, and determine that the direction of the third coordinate axis of the registration coordinate system matches the first vector product.

[0191] The directions of the first, second, and third coordinate axes of the registration coordinate system obtained through the above steps can be represented as three unit vectors of the registration coordinate system, namely:

[0192] In a specific example of step S901, the first vector can be constructed using the starting point (i.e., the first curve point) and the ending point (i.e., the last curve point).

[0193] In a specific example of step S902, the eigenvector corresponding to the largest eigenvalue can be selected for a dot product operation to obtain the first dot product. If the dot product of two vectors is greater than zero, it means that their directions are the same or similar (e.g., within the range of -90° to +90°). Otherwise, the eigenvector is inverted to obtain the direction of the first coordinate axis, which can be represented as a unit vector.

[0194] In a specific example of step S903, the starting point (i.e., the first first curve point) and ending point (i.e., the last first curve point) of the alignment points are selected to calculate the first midpoint, and the midpoint of the curve alignment portion (i.e., the first midpoint) is used to construct the second vector. Furthermore, in step S905, the direction of the second coordinate axis can be determined in a manner similar to that in step S902, thereby obtaining the direction of the second coordinate axis, which can be represented as a unit vector.

[0195] In the specific scheme of step S905, the direction of the third coordinate axis of the registered coordinate system can be obtained by calculating the first vector product, which can be represented as a unit vector.

[0196] In one embodiment, the eigenvectors of the second covariance matrix include a third eigenvector and a fourth eigenvector; the third eigenvector corresponds to the largest eigenvalue of the second covariance matrix, and the fourth eigenvector corresponds to another eigenvalue of the second covariance matrix after the largest eigenvalue.

[0197] Please refer to Figure 11 The process of determining the template coordinate system based on the eigenvectors of the second covariance matrix may include, for example:

[0198] S1101: Obtain the vector from the first second curve point to the last second curve point in the second curve point set as the first vector;

[0199] S1102: By comparing the first vector with the third eigenvector of the second covariance matrix, the direction of the third eigenvector of the second covariance matrix or the reverse vector of the third eigenvector of the second covariance matrix is ​​selected as the positive direction of the first coordinate axis of the template coordinate system.

[0200] S1103: Calculate the midpoint of the line connecting the first second curve point and the last second curve point in the second curve point set as the second midpoint, calculate the second curve point in the middle position in the second curve point set as the second midpoint, and obtain the second vector from the second midpoint to the second midpoint;

[0201] S1104: By comparing the second vector with the fourth eigenvector of the second covariance matrix, the direction of the fourth eigenvector of the second covariance matrix or the reverse vector of the fourth eigenvector of the second covariance matrix is ​​selected as the positive direction of the second coordinate axis of the template coordinate system.

[0202] S1105: Perform a vector product operation on the positive directions of the first coordinate axis and the second coordinate axis of the target coordinate system to obtain a first vector product, and determine that the direction of the third coordinate axis of the target coordinate system matches the first vector product.

[0203] In related technologies, when solving for eigenvectors, the phenomenon that eigenvectors of the covariance matrix have both positive and negative directions is often overlooked. However, if the direction is not determined and adjusted, the registration will correspond to the opposite parts of the point cloud data, resulting in a large registration error. The registration results of two curves can be, for example, [the following is an example of an error]. Figure 10a Figure 10b As shown. To address this deficiency, this application determines whether the coordinate axes of the registration coordinate system and the template coordinate system are aligned with the eigenvectors by checking whether the vector formed by the curve points in the curve to be registered and the template curve are in the same direction (i.e., by determining whether the angle between them is greater than π through the dot product). This ensures the accuracy of the registration result, and the registration result can be, for example... Figure 10c As shown.

[0204] In one embodiment, the pose transformation information is represented as a pose transformation matrix between the curve to be registered and the template curve; the target pose transformation information is represented as a target pose transformation matrix between the curve to be registered and the template curve.

[0205] Please refer to Figure 12 The process of obtaining the target pose transformation information between the curve to be registered and the template curve based on the pose transformation information of the M groups includes:

[0206] S1201: Calculate the pose error information for each group of pose transformation information;

[0207] The pose error information characterizes the pose error between the curve to be registered and the template curve after the pose of the curve to be registered is transformed by the pose transformation information.

[0208] S1202: Based on the pose error information of each group of pose transformation information, determine the weight information of that group of pose error information;

[0209] S1203: Based on the weight information of each group of pose transformation information in the M groups of pose transformation information, the M groups of pose transformation matrices are weighted and summed to obtain the target pose transformation matrix.

[0210] The pose transformation matrix may include, for example, the rotation matrix R between two curves. step Translation matrix t step In one example, the rotation matrix R step Translation matrix t step It can be calculated using the following formula:

[0211]

[0212] t step =<β j >-R step *<αj >

[0213] Using the obtained rotation and translation matrices, the set of the first feature points of the curve to be registered is determined. After the transformation, the set of transformed points and the second characteristic points of the template curve can be calculated. The error at the point in the middle, thus realizing the error ω step The calculation process can be illustrated as follows:

[0214]

[0215] For M point sets obtained according to different scales, the error corresponding to each point set can be obtained, forming an error set. And the transformation matrix corresponding to each point set, forming a set of transformation matrices. Wherein, the rotation matrix R step Translation matrix t step It is described by a transformation matrix T;

[0216] Then, in step S1202, a weighted value can be calculated based on this error. It is understood that the larger the error, the lower the confidence level of the corresponding transformation matrix. Therefore, the weights can be set to be inversely correlated with the error. For example, the "reciprocal of the pose error information" for each group of pose transformation information can be used as the weight value for that group of poses. Then, in step S1203, the weighted sum of multiple groups of transformation matrices is calculated to obtain the multi-scale global transformation matrix T. init (i.e., target pose transformation information).

[0217] In the above scheme, the integration of M groups of pose transformation information is achieved by determining the weight based on the error. Therefore, the larger the error of the pose transformation information, the smaller its contribution to the integrated calculation result of the target pose transformation information, and the smaller the error of the pose transformation information, the greater its contribution to the integrated calculation result of the target pose transformation information. On this basis, the accuracy of the target pose transformation information can be effectively improved.

[0218] In one embodiment, the transformation matrix with the smallest error value can also be directly used as the target pose transformation information;

[0219] Alternatively, the target pose transformation information can be determined based on transformation matrices whose error values ​​meet preset conditions (e.g., error less than a threshold, a specified number of small errors, etc.). For example, the weighted sum of transformation matrices that meet preset conditions can be used as the target pose transformation information, where the weights can be determined based on the error (e.g., weights are inversely correlated with the error), or they can be preset values, etc.

[0220] Figure 13 The process of step S1301 shown can be referred to Figure 2The steps S203 shown are understood, and the same or similar content will not be repeated here.

[0221] After step S1301, the following is also included:

[0222] S1302: Based on the target pose transformation matrix, the final pose transformation matrix is ​​solved using the ICP algorithm.

[0223] The target pose transformation matrix can be characterized as T init ;

[0224] In one embodiment, after determining the target pose transformation matrix according to the above method, it can be determined that the registration of the curve to be registered is completed, that is, the target pose transformation information of the curve to be registered and the template curve is determined; or, in another embodiment, the obtained target pose transformation matrix can be further optimized.

[0225] For example, the target pose transformation matrix can be used as initialization to perform ICP (Iterative ClosestPoint) nearest neighbor iteration to calculate a more accurate transformation matrix T. result (i.e., the final pose transformation matrix). The specific implementation of the ICP algorithm can be found in relevant technologies, and will not be elaborated upon here.

[0226] By introducing the ICP algorithm described above, the accuracy of the final pose transformation matrix can be effectively improved.

[0227] Please refer to Figure 14 This invention also provides a curve registration processing apparatus 1400, comprising:

[0228] The acquisition module 1401 is used to acquire M sets of points, each set of points including a first set of points used to identify the curve to be registered and a second set of points used to identify the template curve; where M is an integer greater than or equal to 1.

[0229] The pose transformation determination module 1402 is used to determine a set of pose transformation information between the curve to be registered and the template curve for each of the M sets of points, based on the first set of points and the second set of points in the set of points respectively.

[0230] The target pose determination module 1403 is used to obtain the target pose transformation information between the curve to be registered and the template curve based on the pose transformation information of the M groups.

[0231] Optionally, the acquisition module 1401 is specifically used for:

[0232] Sampling the curve points of the curve to be registered and the curve points of the template curve;

[0233] According to different scale requirements, the curve points in the curve to be registered and / or the template curve are interpolated M times to obtain new curve points for the corresponding curve. After each interpolation, a first set of curve points of the curve to be registered and a second set of curve points of the template curve are formed. The scale requirement represents the required value or range of distance between adjacent curve points in the first set and the second set.

[0234] Optionally, the pose transformation determination module 1402 is specifically used for:

[0235] For each of the M sets of points, a set of pose transformation information between the curve to be registered and the template curve is determined based on the first and second point sets in that set, including:

[0236] N1 curve points are selected from the first point set as N1 first feature points, and N2 curve points are selected from the second point set as N2 second feature points; where N1 and N2 are both integers greater than or equal to 2.

[0237] Based on the N1 first feature points and N2 second feature points, the target alignment relationship between the first point set and the second point set in the set of points is determined. The target alignment relationship represents that A first curve points in the first point set and A second curve points in the second point set are mutually matched.

[0238] The pose transformation information is determined by using the first and second curve points that match each other.

[0239] Optionally, the pose transformation determination module 1402 is specifically used for:

[0240] First feature information of each first curve point in the first point set is extracted, and N1 first feature points are selected in the first point set based on the first feature information; the first feature information characterizes the curvature and / or torsion of the curve to be registered at the corresponding first curve point.

[0241] Extract the second feature information of each second curve point in the second point set, and select the N2 second feature points in the second point set based on the second feature information; the second feature information characterizes the curvature and / or torsion of the template curve at the corresponding second curve point.

[0242] Optionally, the pose transformation determination module 1402 is specifically used for:

[0243] In the first set of points, the N1 curve points with the highest values ​​of the first feature information are selected as the N1 first feature points;

[0244] In the second set of points, the N2 curve points with the highest values ​​of the second feature information are selected as the N2 first feature points.

[0245] Optionally, the pose transformation determination module 1402 is specifically used for:

[0246] Based on the N1 first feature points and N2 second feature points, multiple candidate alignment relationships between the first point set and the second point set in the set of points are determined. The candidate alignment relationship represents that several first curve points in the first point set and several second curve points in the second point set are matched with each other.

[0247] For each candidate alignment relationship, the feature error between each pair of matching first curve points and second curve points is calculated, and based on the feature error between each pair of matching first curve points and second curve points, the feature error evaluation information under the candidate alignment relationship is determined; wherein, the feature error is determined based on the first feature information of the first curve point and the second feature information of the second curve point.

[0248] Based on the feature error evaluation information of each of the multiple candidate alignment relationships, one candidate alignment relationship is selected as the target alignment relationship from the multiple candidate alignment relationships.

[0249] Optionally, the pose transformation determination module 1402 is specifically used for:

[0250] For each of the N1 first feature points, a second feature point that matches the first feature point is determined from the N2 second feature points, so as to determine the mutually matching first feature point and second feature point; the feature error of the mutually matching first feature point and second feature point is less than a preset threshold.

[0251] in:

[0252] After each pair of matching first feature points and second feature points is determined, the first feature point in the matching feature point pair is taken as the first reference point and the second feature point is taken as the second reference point; then it is determined whether the adjacent first curve points of the first reference point in the first set and the adjacent second curve points of the second reference point in the second set are matching.

[0253] If the adjacent first curve point and the adjacent second curve point match each other, then the adjacent first curve point is taken as the first reference point, the second curve point is taken as the second reference point, and the process returns to the step of determining whether the adjacent first curve feature points of the first reference point and the adjacent second curve points of the second point match each other.

[0254] If the adjacent first curve point and the adjacent second curve point do not match, a candidate alignment relationship is determined based on all the determined mutually matching curve point pairs.

[0255] Optionally, the pose transformation determination module 1402 is specifically used for:

[0256] Among the various candidate alignment relationships, the candidate alignment relationship with the smallest feature error evaluation information value is selected as the target alignment relationship;

[0257] or,

[0258] Among the various candidate alignment relationships, several candidate alignment relationships that satisfy the preset conditions for feature error evaluation information are selected, and the candidate alignment relationship with the most mutually matching curve point pairs is taken as the target alignment relationship; the mutually matching feature point pairs include a first feature point and a second feature point that match each other.

[0259] Optionally, the pose transformation determination module 1402 is specifically used for:

[0260] The pose transformation information is determined using mutually matched first and second curve points, including:

[0261] Using the first and second curve points that match each other, the pose transformation relationship between the registration coordinate system based on the curve to be registered and the template coordinate system based on the template curve is calibrated, and the pose transformation relationship is used as the pose transformation information.

[0262] The registration coordinate system is determined through the following process:

[0263] Obtain the set of first curve points formed by the A first curve points;

[0264] Calculate the first covariance matrix of the first set of curve points;

[0265] Determine the eigenvalues ​​of the first covariance matrix, and based on the eigenvalues ​​of the first covariance matrix, determine the eigenvectors of the first covariance matrix;

[0266] The registration coordinate system is determined based on the eigenvectors of the first covariance matrix;

[0267] The template coordinate system is determined through the following process:

[0268] Obtain the set of second curve points formed by the A second curve points;

[0269] Calculate the second covariance matrix of the second set of curve points;

[0270] Determine the eigenvalues ​​of the second covariance matrix, and based on the eigenvalues ​​of the second covariance matrix, determine the eigenvectors of the second covariance matrix;

[0271] The template coordinate system is determined based on the eigenvectors of the second covariance matrix.

[0272] The eigenvectors of the first covariance matrix include a first eigenvector and a second eigenvector; the first eigenvector corresponds to the largest eigenvalue of the first covariance matrix, and the second eigenvector corresponds to another eigenvalue of the first covariance matrix after the largest eigenvalue.

[0273] Determining the registration coordinate system based on the eigenvectors of the first covariance matrix includes:

[0274] Obtain the vector from the first first curve point to the last first curve point in the first set of curve points as the first vector;

[0275] By comparing the first vector with the first eigenvector of the first covariance matrix, the direction of the first eigenvector of the first covariance matrix or the inverse vector of the first eigenvector of the first covariance matrix is ​​selected as the positive direction of the first coordinate axis of the registration coordinate system.

[0276] The first midpoint is calculated as the midpoint of the line connecting the first first curve point and the last first curve point in the first set of curve points. The first curve point in the middle position in the first set of curve points is calculated as the first midpoint, and the second vector from the first midpoint to the first midpoint is obtained.

[0277] By comparing the second vector with the second eigenvector of the first covariance matrix, the direction of the second eigenvector of the first covariance matrix or the inverse vector of the second eigenvector of the first covariance matrix is ​​selected as the positive direction of the second coordinate axis of the registration coordinate system.

[0278] Perform a vector product operation on the positive directions of the first and second coordinate axes of the registration coordinate system to obtain a first vector product, and determine that the direction of the third coordinate axis of the registration coordinate system matches the first vector product.

[0279] Optionally, by comparing the first vector with the first eigenvector of the first covariance matrix, the direction of the first eigenvector of the first covariance matrix or the inverse vector of the first eigenvector of the first covariance matrix is ​​selected as the positive direction of the first coordinate axis of the registration coordinate system, including:

[0280] Perform a dot product operation between the first vector and the first eigenvector of the first covariance matrix to obtain the first dot product;

[0281] If the first dot product is greater than zero, then the direction of the first eigenvector of the first covariance matrix is ​​determined as the positive direction of the first coordinate axis of the registration coordinate system;

[0282] If the first dot product is not greater than zero, then the direction of the inverse vector of the first eigenvector of the first covariance matrix is ​​determined as the positive direction of the first coordinate axis of the registration coordinate system.

[0283] Optionally, by comparing the second vector with the second eigenvector of the first covariance matrix, the direction of the second eigenvector of the first covariance matrix or the inverse vector of the second eigenvector of the first covariance matrix is ​​selected as the positive direction of the second coordinate axis of the registration coordinate system, including:

[0284] Perform a dot product operation between the second vector and the second eigenvector of the first covariance matrix to obtain the second dot product;

[0285] If the second dot product is greater than zero, then the direction of the second eigenvector of the first covariance matrix is ​​determined as the positive direction of the second coordinate axis of the registration coordinate system;

[0286] If the second dot product is not greater than zero, then the direction of the inverse vector of the second eigenvector of the first covariance matrix is ​​determined as the positive direction of the second coordinate axis of the registration coordinate system.

[0287] Optionally, the eigenvectors of the second covariance matrix include a third eigenvector and a fourth eigenvector; the third eigenvector corresponds to the largest eigenvalue of the second covariance matrix, and the fourth eigenvector corresponds to another eigenvalue of the second covariance matrix after the largest eigenvalue.

[0288] Determining the template coordinate system based on the eigenvectors of the second covariance matrix includes:

[0289] Obtain the vector from the first second curve point to the last second curve point in the second set of curve points as the first vector;

[0290] By comparing the first vector with the third eigenvector of the second covariance matrix, the direction of the third eigenvector of the second covariance matrix or the reverse vector of the third eigenvector of the second covariance matrix is ​​selected as the positive direction of the first coordinate axis of the template coordinate system.

[0291] Calculate the midpoint of the line connecting the first second curve point and the last second curve point in the second curve point set as the second midpoint, calculate the second curve point in the middle position in the second curve point set as the second midpoint, and obtain the second vector from the second midpoint to the second midpoint.

[0292] By comparing the second vector with the fourth eigenvector of the second covariance matrix, the direction of the fourth eigenvector of the second covariance matrix or the reverse vector of the fourth eigenvector of the second covariance matrix is ​​selected as the positive direction of the second coordinate axis of the template coordinate system.

[0293] Perform a vector product operation on the positive directions of the first and second coordinate axes of the target coordinate system to obtain a first vector product, and determine that the direction of the third coordinate axis of the target coordinate system matches the first vector product.

[0294] Optionally, the pose transformation information is represented as the pose transformation matrix between the curve to be registered and the template curve; the target pose transformation information is represented as the target pose transformation matrix between the curve to be registered and the template curve.

[0295] Optionally, the target pose determination module 1403 is specifically used for:

[0296] Calculate the pose error information for each set of pose transformation information. The pose error information represents the pose error between the curve to be registered and the template curve after the pose of the curve to be registered is transformed by the pose transformation information of that set.

[0297] Based on the pose error information of each group of pose transformation information, determine the weight information of that group of pose error information.

[0298] Based on the weight information of each group of pose transformation information in the M groups, the M groups of pose transformation matrices are weighted and summed to obtain the target pose transformation matrix.

[0299] Optionally, the target pose determination module 1403 is specifically used for:

[0300] The pose error information of each group of pose transformation information is used as the weight information of that group of pose error information.

[0301] Figure 15 The curve registration processing device 1500, acquisition module 1501, pose transformation determination module 1502, and target pose determination module 1503 in the illustrated embodiment can be referred to as follows: Figure 14 The curve registration processing device 1400, acquisition module 1401, pose transformation determination module 1402, and target pose determination module 1403 in the illustrated embodiment are understood. For the same or similar content, they will not be described again here.

[0302] The target pose transformation information is represented as the target pose transformation matrix between the curve to be registered and the template curve;

[0303] Please refer to Figure 15The curve registration processing device 1500 also includes:

[0304] The final pose determination module 1504 is used to solve the final pose transformation matrix based on the target pose transformation matrix using the ICP algorithm.

[0305] Please refer to Figure 16 An electronic device 1600 is provided, comprising:

[0306] Processor 1601; and,

[0307] Memory 1602 is used to store the executable instructions of the processor;

[0308] The processor 1601 is configured to execute the methods described above by executing the executable instructions.

[0309] The processor 1601 can communicate with the memory 1602 via the bus 1603.

[0310] This invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the methods described above.

[0311] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0312] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for curve registration, characterized in that, include: Obtain M sets of points, each set of points including a first set of points used to identify the curve to be registered, and a second set of points used to identify the template curve; where M is an integer greater than or equal to 1; For each of the M sets of points, determine a set of pose transformation information between the curve to be registered and the template curve; Based on the pose transformation information of the M groups, the target pose transformation information between the curve to be registered and the template curve is obtained; Obtain M sets of points, including: Sampling the curve points of the curve to be registered and the curve points of the template curve; According to different scale requirements, interpolation is performed M times on the curve points in the curve to be registered and / or the template curve to obtain new curve points for the corresponding curve. After each interpolation, a first set of curve points for the curve to be registered and a second set of curve points for the template curve are formed. The scale requirement characterizes the required value or range of distance between adjacent curve points in the first set and the second set. in: The registration curves characterize the shape of the bronchoscope used to insert into the bronchial tree. The template curve characterizes the shape of a tracheal segment in the bronchial tree.

2. The curve registration processing method according to claim 1, characterized in that, For each of the M sets of points, determine a set of pose transformation information between the curve to be registered and the template curve, including: N1 curve points are selected from the first point set as N1 first feature points, and N2 curve points are selected from the second point set as N2 second feature points; where N1 and N2 are both integers greater than or equal to 2. Based on the N1 first feature points and N2 second feature points, the target alignment relationship between the first point set and the second point set in the set of points is determined. The target alignment relationship represents that A first curve points in the first point set and A second curve points in the second point set are mutually matched. The pose transformation information is determined by using the first and second curve points that match each other.

3. The curve registration processing method according to claim 2, characterized in that, N1 curve points are selected from the first point set as N1 first feature points, and N2 curve points are selected from the second point set as N2 second feature points, including: First feature information of each first curve point in the first point set is extracted, and N1 first feature points are selected in the first point set based on the first feature information; the first feature information characterizes the curvature and / or torsion of the curve to be registered at the corresponding first curve point. Extract the second feature information of each second curve point in the second point set, and select the N2 second feature points in the second point set based on the second feature information; the second feature information characterizes the curvature and / or torsion of the template curve at the corresponding second curve point.

4. The curve registration processing method according to claim 3, characterized in that, Based on the first feature information, selecting the N1 first feature points from the first point set includes: selecting the N1 first curve points with the highest values ​​of the first feature information from the first point set as the N1 first feature points; Based on the second feature information, selecting the N2 second feature points from the second point set includes: selecting the N2 second curve points with the highest values ​​of the second feature information from the second point set as the N2 second feature points.

5. The curve registration processing method according to claim 2, characterized in that, Based on the N1 first feature points and N2 second feature points, the target alignment relationship between the first point set and the second point set in this set of points is determined, including: Based on the N1 first feature points and N2 second feature points, multiple candidate alignment relationships between the first point set and the second point set in the set of points are determined. The candidate alignment relationship represents that several first curve points in the first point set and several second curve points in the second point set are matched with each other. For each candidate alignment relationship, the feature error between each pair of matching first curve points and second curve points is calculated, and based on the feature error between each pair of matching first curve points and second curve points, the feature error evaluation information under the candidate alignment relationship is determined; wherein, the feature error is determined based on the first feature information of the first curve point and the second feature information of the second curve point. Based on the feature error evaluation information of each of the multiple candidate alignment relationships, one candidate alignment relationship is selected as the target alignment relationship from the multiple candidate alignment relationships.

6. The curve registration processing method according to claim 5, characterized in that, Based on the N1 first feature points and N2 second feature points, various candidate alignment relationships between the first point set and the second point set in this set of points are determined, including: For each of the N1 first feature points, a second feature point that matches the first feature point is determined from the N2 second feature points to determine the mutually matching first feature point and second feature point; the feature error of the mutually matching first feature point and second feature point is less than a preset threshold. in: After each pair of matching first feature points and second feature points is determined, the first feature point in the matching feature point pair is taken as the first reference point and the second feature point is taken as the second reference point; then it is determined whether the adjacent first curve points of the first reference point in the first set and the adjacent second curve feature points of the second reference point in the second set are mutually matched. If the adjacent first curve point and the adjacent second curve point match each other, then the adjacent first curve point is taken as the first reference point and the second curve point is taken as the second reference point, and the process returns to the step of determining whether the adjacent first curve points of the first reference point in the first set and the adjacent second curve feature points of the second reference point in the second set match each other. If the adjacent first curve point and the adjacent second curve point do not match, a candidate alignment relationship is determined based on all the determined mutually matching curve point pairs.

7. The curve registration processing method according to claim 5, characterized in that, Based on the feature error evaluation information of each of the multiple candidate alignment relationships, one candidate alignment relationship is selected as the target alignment relationship from the multiple candidate alignment relationships, including: Among the various candidate alignment relationships, the candidate alignment relationship with the smallest feature error evaluation information value is selected as the target alignment relationship; or, Among the various candidate alignment relationships, several candidate alignment relationships that satisfy the preset conditions for feature error evaluation information are selected, and the candidate alignment relationship with the most mutually matching curve point pairs is taken as the target alignment relationship; the mutually matching curve point pairs include a first curve point and a second curve point that match each other.

8. The curve registration processing method according to any one of claims 2 to 7, characterized in that, The pose transformation information is determined using mutually matched first and second curve points, including: Using the first and second curve points that match each other, the pose transformation relationship between the registration coordinate system based on the curve to be registered and the template coordinate system based on the template curve is calibrated, and the pose transformation relationship is used as the pose transformation information.

9. The curve registration processing method according to claim 8, characterized in that, The registration coordinate system is determined through the following process: Obtain the set of first curve points formed by the A first curve points; Calculate the first covariance matrix of the first set of curve points; Determine the eigenvalues ​​of the first covariance matrix, and based on the eigenvalues ​​of the first covariance matrix, determine the eigenvectors of the first covariance matrix; The registration coordinate system is determined based on the eigenvectors of the first covariance matrix; The template coordinate system is determined through the following process: Obtain the set of second curve points formed by the A second curve points; Calculate the second covariance matrix of the second set of curve points; Determine the eigenvalues ​​of the second covariance matrix, and based on the eigenvalues ​​of the second covariance matrix, determine the eigenvectors of the second covariance matrix; The template coordinate system is determined based on the eigenvectors of the second covariance matrix.

10. The curve registration processing method according to claim 9, characterized in that, The eigenvectors of the first covariance matrix include a first eigenvector and a second eigenvector; the first eigenvector corresponds to the largest eigenvalue of the first covariance matrix, and the second eigenvector corresponds to another eigenvalue of the first covariance matrix after the largest eigenvalue. Determining the registration coordinate system based on the eigenvectors of the first covariance matrix includes: Obtain the vector from the first first curve point to the last first curve point in the first set of curve points as the first vector; By comparing the first vector with the first eigenvector of the first covariance matrix, the direction of the first eigenvector of the first covariance matrix or the inverse vector of the first eigenvector of the first covariance matrix is ​​selected as the positive direction of the first coordinate axis of the registration coordinate system. The first midpoint is calculated as the midpoint of the line connecting the first first curve point and the last first curve point in the first set of curve points. The first curve point in the middle position in the first set of curve points is calculated as the first midpoint, and the second vector from the first midpoint to the first midpoint is obtained. By comparing the second vector with the second eigenvector of the first covariance matrix, the direction of the second eigenvector of the first covariance matrix or the inverse vector of the second eigenvector of the first covariance matrix is ​​selected as the positive direction of the second coordinate axis of the registration coordinate system. Perform a vector product operation on the positive directions of the first and second coordinate axes of the registration coordinate system to obtain a first vector product, and determine that the direction of the third coordinate axis of the registration coordinate system matches the first vector product.

11. The curve registration processing method according to claim 10, characterized in that, By comparing the first vector with the first eigenvector of the first covariance matrix, the direction of the first eigenvector of the first covariance matrix or the inverse vector of the first eigenvector of the first covariance matrix is ​​selected as the positive direction of the first coordinate axis of the registration coordinate system, including: Perform a dot product operation between the first vector and the first eigenvector of the first covariance matrix to obtain the first dot product; If the first dot product is greater than zero, then the direction of the first eigenvector of the first covariance matrix is ​​determined as the positive direction of the first coordinate axis of the registration coordinate system; If the first dot product is not greater than zero, then the direction of the inverse vector of the first eigenvector of the first covariance matrix is ​​determined as the positive direction of the first coordinate axis of the registration coordinate system.

12. The curve registration processing method according to claim 10, characterized in that, By comparing the second vector with the second eigenvector of the first covariance matrix, the direction of the second eigenvector of the first covariance matrix or the inverse vector of the second eigenvector of the first covariance matrix is ​​selected as the positive direction of the second coordinate axis of the registration coordinate system, including: Perform a dot product operation between the second vector and the second eigenvector of the first covariance matrix to obtain the second dot product; If the second dot product is greater than zero, then the direction of the second eigenvector of the first covariance matrix is ​​determined as the positive direction of the second coordinate axis of the registration coordinate system; If the second dot product is not greater than zero, then the direction of the inverse vector of the second eigenvector of the first covariance matrix is ​​determined as the positive direction of the second coordinate axis of the registration coordinate system.

13. The curve registration processing method according to claim 9, characterized in that, The eigenvectors of the second covariance matrix include a third eigenvector and a fourth eigenvector; the third eigenvector corresponds to the largest eigenvalue of the second covariance matrix, and the fourth eigenvector corresponds to another eigenvalue of the second covariance matrix after the largest eigenvalue. Determining the template coordinate system based on the eigenvectors of the second covariance matrix includes: Obtain the vector from the first second curve point to the last second curve point in the second set of curve points as the first vector; By comparing the first vector with the third eigenvector of the second covariance matrix, the direction of the third eigenvector of the second covariance matrix or the reverse vector of the third eigenvector of the second covariance matrix is ​​selected as the positive direction of the first coordinate axis of the template coordinate system. Calculate the midpoint of the line connecting the first second curve point and the last second curve point in the second curve point set as the second midpoint, calculate the second curve point in the middle position in the second curve point set as the second midpoint, and obtain the second vector from the second midpoint to the second midpoint. By comparing the second vector with the fourth eigenvector of the second covariance matrix, the direction of the fourth eigenvector of the second covariance matrix or the reverse vector of the fourth eigenvector of the second covariance matrix is ​​selected as the positive direction of the second coordinate axis of the template coordinate system. Perform a vector product operation on the positive directions of the first and second coordinate axes of the target coordinate system to obtain a first vector product, and determine that the direction of the third coordinate axis of the target coordinate system matches the first vector product.

14. The curve registration processing method according to any one of claims 1 to 7, characterized in that, The pose transformation information is represented as the pose transformation matrix between the curve to be registered and the template curve; the target pose transformation information is represented as the target pose transformation matrix between the curve to be registered and the template curve. Based on the pose transformation information of the M groups, the target pose transformation information between the curve to be registered and the template curve is obtained, including: Calculate the pose error information for each set of pose transformation information. The pose error information represents the pose error between the curve to be registered and the template curve after the pose of the curve to be registered is transformed by the pose transformation information of that set. Based on the pose error information of each group of pose transformation information, determine the weight information of that group of pose error information. Based on the weight information of each group of pose transformation information in the M groups, the M groups of pose transformation matrices are weighted and summed to obtain the target pose transformation matrix.

15. A curve registration processing device, characterized in that, include: The acquisition module is used to acquire M sets of points. Each set of points includes a first set of points used to identify the curve to be registered and a second set of points used to identify the template curve; where M is an integer greater than or equal to 1. The pose transformation determination module is used to determine a set of pose transformation information between the curve to be registered and the template curve for each of the M sets of points. The target pose determination module is used to obtain the target pose transformation information between the curve to be registered and the template curve based on the M sets of pose transformation information; Obtain M sets of points, including: Sampling the curve points of the curve to be registered and the curve points of the template curve; According to different scale requirements, the curve points in the curve to be registered and / or the template curve are interpolated M times to obtain new curve points for the corresponding curve. After each interpolation, a first set of curve points of the curve to be registered and a second set of curve points of the template curve are formed. The scale requirement represents the required value or range of distance between adjacent curve points in the first set and the second set. in: The registration curves characterize the shape of the bronchoscope used to insert into the bronchial tree. The template curve characterizes the shape of a tracheal segment in the bronchial tree.

16. An electronic device, characterized in that, Including processor and memory, The memory is used to store code; The processor is configured to execute code in the memory to implement the method according to any one of claims 1 to 14.

17. A storage medium having a computer program stored thereon, which, when executed by a processor, implements the method of any one of claims 1 to 14.

Images