A method for multi-linear deformable body tracking based on structure preserving registration and iterative geometric decoupling

Abstract

The application provides a kind of based on structure keeping registration and iterative geometric separation multilinear deformable body tracking method, comprising: step one, using structure keeping registration model extraction and track the feature points of multilinear deformable body, according to the feature point construction feature list;Step two, using geometric method separates out n linear deformable body, according to the linear deformable body separates the feature points in the feature list into n list, wherein n is the number of linear deformable body;Step three, robot combines each feature point in n list and independently operates multilinear deformable body;Step four, judge whether robot independently operates multilinear deformable body is completed, if complete, then task ends, otherwise go to step one.The application based on structure keeping registration and iterative geometric separation carries out whole tracking to multilinear deformable body simultaneously, also separates each linear deformable body in real time, realizes the tracking of each linear deformable body respectively.

Description

Technical Field

[0001] This invention relates to the field of machine vision tracking, and more particularly to a method for tracking multilinear deformable bodies based on structure-preserving registration and iterative geometric separation. Background Technology

[0002] Linear deformable bodies (DLOs) are objects whose appearance is linear and whose shape and size continuously change with external operations, such as ropes, cables, and fibers. Machine vision tracking of DLOs mainly involves using optical sensing devices such as cameras to extract the dynamic state features of each DLO in real time and constructing the correspondence between the state features of each DLO at different times, providing guidance for the robot to autonomously operate each DLO.

[0003] When a DLO is subjected to external forces, its continuous deformation behavior exhibits typical characteristics of "infinite degrees of freedom." This means that detection points with distinct structural features (such as inflection points and corners) within the DLO cannot maintain their relative positions within the object at different times, unlike a rigid body. Therefore, DLOs exhibit irregular shapes and difficulties in deformation tracking. The following research has been conducted to explore how to track the states of DLOs, which differ from those of rigid bodies.

[0004] Machine vision tracking for DLO (Direct Response) primarily provides the perceptual basis for autonomous DLO operations of robots and is one of the core components of DLO operation. For example, in the autonomous operation of robots handling electrical control cabinet cable routing and hydraulic hose arrangement, it provides feedback to the robot on the position and attitude changes of cables and hydraulic hoses under the robot's operation, guiding the robot to take appropriate actions in response to these changes.

[0005] The invention patent publication with publication number CN116433718A discloses a feature tracking system and method. This document describes a point tracking method that uses a 3DMM model to detect and track deformable objects (such as the human body) based on their obvious features (such as eyebrows, nose, and ears), thereby achieving motion capture of the deformable objects. However, it is not suitable for tracking linear deformable bodies without obvious features.

[0006] The invention patent publication CN1908963A discloses a method for detecting and tracking deformable objects. This patent document is used to detect and track deformable objects with continuously changing characteristics. It infers the shape vector sequence by maximizing the probability of a time statistical shape model with continuously changing embedding functions, thereby achieving the tracking of deformable objects. Its main function is to segment moving deformable objects from noisy backgrounds, rather than focusing on matching the features of the deformable object at different times. Furthermore, it is only applicable to tracking a single deformable object and cannot achieve separate tracking of multiple linear deformable objects.

[0007] The invention patent publication CN116235075A discloses a system and method for tracking deformation. This system senses moving deformable objects in a scene and tracks the nominal shape deformation of objects in the scene through calibration matching between the tracking system and the measurement system. However, this patent document is only applicable to tracking a single deformable body and cannot achieve separate tracking of multiple linear deformable bodies.

[0008] In modern industrial automation and household applications, tracking multiple DLOs (Device Locators and Wires) is often more common. In industrial automation, for example, when inspecting electrical control cabinets or hydraulic systems, operators or robots typically encounter scenarios where multiple cables or hydraulic hoses intersect irregularly. In household applications, for example, when bundling, knotting, or braiding ropes or fibers, operators or robots often face scenarios where multiple ropes or fibers intersect irregularly. These irregular and difficult-to-track characteristics of DLOs are intertwined and interfere with each other, yet operators or robots need to track them separately to perform individual operations, presenting new challenges to DLO tracking.

[0009] While adding multi-DLO separation preprocessing before DLO tracking is a feasible approach, traditional threshold-based, edge-based, and region-based image segmentation techniques cannot effectively separate multiple DLOs when they look identical or similar, such as when tracking cables of the same color and diameter. Furthermore, using deep learning-based image segmentation methods for multi-DLO separation requires large datasets for training, resulting in high costs. Summary of the Invention

[0010] The purpose of this invention is to provide a multilinear deformable body tracking method based on structure-preserving registration and iterative geometric separation. This method is embedded in the autonomous operation process of a robot, enabling the robot to perform mutual separation and tracking while operating multiple DLOs, and iteratively guiding the robot's autonomous operation.

[0011] The technical solution for achieving the objective of this invention is as follows:

[0012] This invention provides a method for tracking multilinear deformable bodies based on structure-preserving registration and iterative geometric separation, comprising:

[0013] Step 1: Extract and track feature points of the multilinear deformable body using a structure-preserving registration model, and construct a feature list based on the feature points;

[0014] Step 2: Separate n linear deformable bodies using geometric methods, and then separate the feature points in the feature list into n lists based on the linear deformable bodies, where n is the number of linear deformable bodies;

[0015] Step 3: The robot autonomously manipulates the multilinear deformable body by combining the feature points from the n lists;

[0016] Step 4: Determine whether the robot has completed its autonomous operation of the multilinear deformable body. If it has, the task ends; otherwise, proceed to Step 1.

[0017] As a further improvement of the present invention, step one includes:

[0018] An original image is obtained by acquiring an image of a scene containing multiple linear deformable bodies, and the original image is then processed to obtain a grayscale image.

[0019] The multilinear deformable body and the background in the grayscale image are segmented to obtain a list of the positions of each pixel of the multilinear deformable body.

[0020] Feature points are obtained by extracting and tracking the location list using a structure-preserving registration model, and a feature list is constructed based on the coordinates of the feature points.

[0021] As a further improvement of the present invention, step one specifically includes:

[0022] A scene of arbitrarily intersecting but independent multilinear deformable bodies is captured using a camera to obtain an original image I, which is then converted into a grayscale image I. gray ;

[0023] The grayscale image I is processed using a color threshold-based mask. gray The multilinear deformable body is segmented from the background to obtain a list of the positions of all multilinear deformable body pixels. def , represented as:

[0024]

[0025] in, The symbol represents the Hadamard product operation, which multiplies the values ​​at corresponding positions of two matrices of the same size; M represents a mask based on a color threshold.

[0026] The location list I is extracted and tracked using a structure-preserving registration model. def All feature points of a multilinear deformable body are used to construct a feature list D based on the coordinates of the feature points.

[0027] D = (d1, d2, ..., d n )=((u1,v1),(u2,v2),...(u n ,v n ))

[0028] Where D represents the feature list, d represents the feature point in the list, and (u,v) represents the x and y coordinates of the corresponding position of the feature point.

[0029] As a further improvement of the present invention, step two includes:

[0030] Extract the cross candidate points of the multilinear deformable body and construct a cross candidate point list; extract the cross points based on the cross candidate points and construct the cross point list.

[0031] Based on the feature list and the intersection point list from step one, feature points belonging to the i-th multilinear deformable body are separated to construct list D. i This yields n lists.

[0032] As a further improvement of the present invention, the step of extracting the cross candidate points of the multilinear deformable body and constructing a cross candidate point list, and extracting cross points based on the cross candidate points and constructing the cross point list, includes:

[0033] Construct a KD-tree from all elements in the feature list described in step one, and query each element d in the KD-tree. i The distances of a adjacent elements from element d are respectively i distance d i1 d i2 , ...,d ia ;

[0034] Find the densest points in the feature list, and assign each element d... i d i1 d i2 , ...,d ia The sum of each element is obtained by adding them together. The m elements with the smallest sum are then used to construct a list of cross-crossing candidate points C, where m represents the number of cross-crossing candidate points.

[0035] The k-means clustering method is used to cluster the candidate cross-point list C to obtain cluster centers. These cluster centers are then used to construct the cross-point list Cr, and each element in the candidate cross-point list C is separated according to its respective cluster center. In this embodiment of the invention, the candidate cross-point list C and the cross-point list Cr are two different lists. The candidate cross-point list C is a list of feature points that meet certain conditions, and its contents are the x and y coordinates of these feature points. The cross-point list Cr is the list obtained by performing clustering operations on the candidate cross-point list C, and its contents are the x and y coordinates of each cluster center.

[0036] As a further improvement of the present invention, the feature points belonging to the i-th multilinear deformable body are separated from the feature list and the intersection point list in step one to construct a feature point list D. i ,include:

[0037] Randomly select a feature point d from the feature list described in step one. i Starting from the point, find the connection point d along the line in a forward direction. k ;

[0038] Determine the connection point d k If the distance from the starting point is less than a threshold, then determine the connection point d. k If the number of secondary connection points exceeds a threshold, then a list D is constructed using the feature points found along the current line. i ;

[0039] Based on connection point d k Find secondary connection points, or after swapping the initial and connection points used initially, search for secondary connection points along the line in reverse. If a secondary connection point exists, select a suitable secondary connection point, replace the initial point with the connection point, and replace the connection point with the secondary connection point. If no secondary connection point exists, separate the feature points found along the line in this round of searching for connections, until a list D of feature points belonging to the i-th multilinear deformable body is constructed from the feature list and the intersection point list. i .

[0040] As a further improvement of the present invention, a feature point d is randomly selected from the feature list in step one. i Starting from the point, find the connection point d along the line in a forward direction. k Specifically:

[0041] Randomly select a feature point d from the feature list described in step one. i As the starting point of the search along the line, this search along the line is denoted as a forward search along the line;

[0042] Determine feature point d i If a feature point belongs to the list of crossover candidate points, then feature point d is used. i The corresponding cluster center Cr j Replace feature point d with coordinates i And remove Cr from the feature list j The cluster centers are all elements in the list C; if an element does not belong, the cluster center is represented by its feature point d. i As the coordinates of the starting point for searching along the line, delete feature point d from the feature list. i Find feature point d in the feature list based on the starting point coordinates. k .

[0043] As a further improvement of the present invention, the connection point d is determined. k If the distance from the starting point is less than a threshold, then determine the connection point d. k If the number of secondary connection points exceeds a threshold, then a list D is constructed using the feature points found along the current line. i According to the connection point d kFind secondary connection points, or after swapping the initial and connection points used initially, search for secondary connection points along the line in reverse. If a secondary connection point exists, select a suitable secondary connection point, replace the initial point with the connection point, and replace the connection point with the secondary connection point. If no secondary connection point exists, separate the feature points found along the line in this round of searching for connections, until a list D of feature points belonging to the i-th multilinear deformable body is constructed from the feature list and the intersection point list. i ;include:

[0044] Based on feature point d k If the distance from the starting point is less than a threshold, then the adjacent feature points d are considered. k Let the starting point be the connection point, and determine the edge from the starting point to the connection point as the starting edge;

[0045] Determine adjacent feature points d k If a point belongs to the list of crossover candidate points, then use the adjacent feature point d. k The corresponding cluster center Cr m Replace feature point d with coordinates k And remove Cr from the feature list m The cluster centers are all elements in the list C; if an element does not belong, the cluster center is represented by its feature point d. k As the coordinates of the connection points searched along the line, feature point d is deleted from the feature list. k Based on the coordinates of the connection point, find the b nearest adjacent elements in the feature list as secondary connection points;

[0046] Exclude secondary connection points whose distance from the connection point exceeds a threshold, and calculate the angle θ constructed between the starting point, connection point, and secondary connection point; construct a score η by weighting the distance between connection points and secondary connection points and the angle θ between the starting point and connection point;

[0047] The connection probability P is constructed by the score η and the number of occurrences τ of the corresponding triple connection group.

[0048] Select the triplet with the highest connection probability P as the connection scheme, and increment its occurrence count by 1.

[0049] Replace the connection point with the new starting point, replace the selected secondary connection point with the new connection point, use the randomly selected point used for the forward line search connection as the connection point, and use the other endpoint of the starting edge of the forward line search as the starting point. This line search is called the reverse line search.

[0050] In this round of searching for connections along the line, elements that appear in the ternary connection group are denoted as feature points of the i-th linear deformable body that has been separated, and list D is constructed. i ;

[0051] Continue until list D is empty or contains only one element. Then, add the deleted element from list C to list D and restart the search for connection along the line of another linear transformation.

[0052] As a further improvement to the present invention, step three, in which the robot autonomously operates the multilinear deformable body by combining the feature points in the n lists, specifically involves:

[0053] Input the feature point information of the linear deformable body and the position and orientation information of the linear deformable body as desired by the user;

[0054] The robot autonomously selects a suitable grasping position and performs actions based on feature point information, the user's desired position information, and posture information, and then executes the operation steps.

[0055] Output the pose information of the multilinear deformable body.

[0056] As a further improvement of the present invention, step four determines whether the robot's autonomous operation of the multilinear deformable body is completed. If it is completed, the task ends; otherwise, it proceeds to step one, including:

[0057] Input the position and attitude information of the linear deformable body as desired by the user, as well as the position and attitude information of the multilinear deformable body output in step three;

[0058] The user's desired location information is compared with the attitude information, and the location information output in step three is compared with the attitude information to obtain the comparison result.

[0059] Based on the comparison results, it is determined whether the robot's operation of the linear deformable body process has been completed, and based on whether it has been completed, it is selected to return to step one or end.

[0060] Compared with the prior art, the beneficial effects of the present invention are:

[0061] 1. This invention is based on a structure-preserving registration and iterative geometric separation method, which is embedded in the process of robot autonomous operation. As the robot autonomously operates on a multilinear deformable body, it simultaneously performs separation and tracking of the multilinear deformable body, thereby achieving mutual separation and tracking while operating multiple DLOs and iteratively guiding the robot's autonomous operation.

[0062] 2. Based on the single linear deformable body tracking method of SPR, this invention combines iterative geometric separation to perform overall tracking of multilinear deformable bodies while separating each linear deformable body in real time, thereby achieving separate tracking of each DLO.

[0063] 3. This invention uses an iterative geometric separation method to separate each linear deformable body, which does not require dataset training, has low cost, is less affected by the appearance similarity of the linear deformable bodies, and has high robustness; this invention embeds dynamic iterative separation in the operation process, which has the advantage of good separation effect. Attached Figure Description

[0064] Figure 1 A flowchart of a multilinear deformable body tracking method based on structure-preserving registration and iterative geometric separation provided by the present invention;

[0065] Figure 2 An example diagram illustrating the application scenario of the multilinear deformable body tracking method based on structure-preserving registration and iterative geometric separation provided by this invention;

[0066] Figure 3 Provided by the present invention Figure 2 A flowchart illustrating a multilinear deformable body tracking method based on structure-preserving registration and iterative geometric separation;

[0067] Figure 4 This is a flowchart of step 2-1 in an embodiment of the present invention;

[0068] Figure 5 This is a flowchart of step 2-2 in an embodiment of the present invention;

[0069] Figure 6 This is a schematic diagram of the construction of the starting edge in step 2-2-3 of this embodiment of the invention;

[0070] Figure 7 This is a schematic diagram of the secondary connection point in step 2-2-5 of this embodiment of the invention;

[0071] Figure 8 This is a schematic diagram illustrating the angle calculation of the starting point, connection point, and secondary connection point in step 2-2-6 of this embodiment of the invention.

[0072] Figure 9 This is a comparison diagram of the position and attitude information of the multilinear deformable body after the robot performs the action in step four of this embodiment of the invention, and the position and attitude information of the multilinear deformable body expected by the user.

[0073] Figure 10 The embodiments of the present invention are based on Figure 9 Example diagram of comparison methods;

[0074] Figure 11 This illustration shows the process of embedding a robot's autonomous operation of a linear deformable body into the multilinear deformable body tracking method provided in this embodiment of the invention. Detailed Implementation

[0075] The present invention will now be described in detail with reference to the embodiments shown in the accompanying drawings. However, it should be noted that these embodiments are not intended to limit the present invention. Equivalent changes or substitutions in function, method, or structure made by those skilled in the art based on these embodiments are all within the scope of protection of the present invention.

[0076] Please see Figure 1 , Figure 1 This invention provides a flowchart of a multilinear deformable body tracking method based on structure-preserving registration and iterative geometric separation, which includes the following steps:

[0077] Step 1: Use a structure-preserving registration model to extract and track feature points of the multilinear deformable body, and construct a feature list based on the feature points.

[0078] Step one of the preferred embodiments of the present invention includes:

[0079] The process involves acquiring images of a scene containing multilinear deformable bodies to obtain the original image, and then processing the original image into grayscale to obtain a grayscale image. Specifically, a camera is used to acquire scenes of arbitrarily intersecting but independent multilinear deformable bodies to obtain the original image I, which is then converted into a grayscale image I. gray .

[0080] The multilinear deformable body in the grayscale image is segmented from the background to obtain a list of the positions of each pixel of the multilinear deformable body; specifically, a color threshold-based mask is used to segment the grayscale image I. gray The multilinear deformable body is segmented from the background to obtain a list of the positions of all multilinear deformable body pixels. def , represented as:

[0081]

[0082] in, represents the Hadamard product operation, which means multiplying the corresponding values ​​of two matrices of the same size; M represents a mask based on a color threshold.

[0083] A structure-preserving registration model is used to extract and track a list of locations to obtain feature points. A feature list is then constructed based on the coordinates of these feature points. Specifically, a structure-preserving registration model is used to extract and track a list of locations. def All feature points of a multilinear deformable body are used to construct a feature list D based on the coordinates of the feature points.

[0084] D = (d1, d2, ..., d n )=((u1,v1),(u2,v2),...(u n ,v n ))

[0085] Where D represents the feature list, d represents the feature point in the list, and (u,v) represents the x and y coordinates of the corresponding position of the feature point.

[0086] In practical applications, step one of the embodiments of the present invention specifically includes: extracting and tracking feature points in a multilinear deformable scene using a structure-preserving registration method, and constructing a list D.

[0087] Step 1-1: Use a camera to capture arbitrarily intersecting but independent multi-DLO scenes, obtain the original image I, and convert it into a grayscale image I. gray .

[0088] Steps 1-2: Apply a color threshold-based mask to the grayscale image I gray The background and deformable body are segmented to obtain a list of the positions of all DLO pixels. def .

[0089] Steps 1-3: Use the SPR method to extract and trace list I def Find all feature points in the DLO (Dual Loop) and construct a list D with the coordinates of these feature points. (n is the number of feature points)

[0090] D = (d1, d2, ..., d n )=((u1,v1),(u2,v2),...(u n ,v n )).

[0091] Step 2: Use a geometric method to separate n linear deformable bodies from the multilinear deformable body. Based on the linear deformable bodies, separate the feature points in the feature list into n lists, where n is the number of linear deformable bodies in the multilinear deformable body.

[0092] Step two of the preferred embodiment of the present invention includes:

[0093] Extract the cross candidate points of the multilinear deformable body and construct a cross candidate point list; extract the cross points based on the cross candidate points and construct a cross point list.

[0094] Based on the feature list and intersection list from step one, feature points belonging to the i-th multilinear deformable body are separated to construct list D. i This yields n lists.

[0095] The embodiments of the present invention describe the extraction of candidate intersection points for multilinear deformable bodies and the construction of a candidate intersection point list. The extraction of intersection points based on the candidate intersection points and the construction of the intersection point list include:

[0096] Construct a KD-tree from all elements in the feature list from step one, and query each element d in the KD-tree. iThe distances of a adjacent elements from element d are respectively i distance d i1 d i2 , ...,d ia ;

[0097] Find the densest points in the feature list and group each element d. i d i1 d i2 , ...,d ia The sum of each element is obtained by adding them together. The m elements with the smallest sum are then used to construct a list of cross-crossing candidate points C, where m represents the number of cross-crossing candidate points.

[0098] The k-means clustering method is used to cluster the cross-point candidate list C to obtain cluster centers. The cluster centers are then used to construct the cross-point list Cr, and each element in the cross-point candidate list C is separated according to its respective cluster center.

[0099] Based on the feature list and intersection list from step one, feature points belonging to the i-th multilinear deformable body are separated to construct list D. i ,include:

[0100] (1) Randomly select a feature point d from the feature list in step one. i Starting from the point, find the connection point d along the line in a forward direction. k Specifically, a feature point d is randomly selected from the feature list in step one. i As the starting point for the search along the line, this search is denoted as a forward search along the line; determine the feature point d. i If a feature point belongs to the list of intersection points, then use feature point d. i The corresponding cluster center Cr j Replace feature point d with coordinates i And remove Cr from the feature list j The cluster centers are all elements in the list C; if an element does not belong, the cluster center is represented by its feature point d. i As the coordinates of the starting point for searching along the line, delete feature point d from the feature list. i Find feature point d in the feature list based on the starting point coordinates. k .

[0101] (2) Determine the connection point d k If the distance from the starting point is less than a threshold, then determine the connection point d. k If the number of secondary connection points exceeds a threshold, then a list D is constructed using the feature points found along the current line. i ;

[0102] (3) Based on the connection point d kFind secondary connection points, or swap the initial and connection points used initially and then search for secondary connection points along the line in reverse. If a secondary connection point exists, select a suitable secondary connection point, replace the initial point with the connection point, and replace the connection point with the secondary connection point. If no secondary connection point exists, separate the feature points that were connected along the line in this round, until feature points belonging to the i-th multilinear deformable body are separated from the feature list and intersection list to construct list D. i .

[0103] Specifically, in this embodiment of the invention, the connection point d is determined. k If the distance from the starting point is less than a threshold, then determine the connection point d. k If the number of secondary connection points exceeds a threshold, then a list D is constructed using the feature points found along the current line. i According to the connection point d k Find secondary connection points, or swap the initial and connection points used initially and then search for secondary connection points along the line in reverse. If a secondary connection point exists, select a suitable secondary connection point, replace the initial point with the connection point, and replace the connection point with the secondary connection point. If no secondary connection point exists, separate the feature points that were connected along the line in this round, until feature points belonging to the i-th multilinear deformable body are separated from the feature list and intersection list to construct list D. i ;include:

[0104] Based on feature point d k If the distance from the starting point is less than a threshold, then the adjacent feature points d are considered. k Let the starting point be the connection point, and determine the edge from the starting point to the connection point as the starting edge;

[0105] Determine adjacent feature points d k If a point belongs to the list of crossover candidate points, then use the adjacent feature point d. k The corresponding cluster center Cr m Replace feature point d with coordinates k And remove Cr from the feature list m The cluster centers are all elements in the list C; if an element does not belong, the cluster center is represented by its feature point d. k As the coordinates of the connection points searched along the line, feature point d is deleted from the feature list. kAccording to the coordinates of the connection point, find the nearest b adjacent elements in the feature list as secondary connection points; it should be noted that (1) when the feature point of this embodiment belongs to the cross candidate point list C, the cross candidate point and the feature points near it are relatively messy, and geometric connection errors are very likely to occur. Therefore, when encountering a feature point belonging to the cross candidate point list in geometric connection, it is necessary to organize the messy feature points and then perform geometric connection. The specific method is: when encountering a feature point belonging to the cross candidate point list C, find the coordinates of the cluster center corresponding to the feature point in the cross point list Cr, replace the coordinates of the feature point, and perform geometric connection; (2) the reason for deleting from the list in this embodiment is: it means that the point has completed geometric connection in this round. If it is not deleted, the optimal connection point of the next point will be the previous point that has not been deleted, causing the connection to fall into an infinite loop between these two points; (3) when the list is empty, it means that all feature points have been connected and the current round of the loop ends; when the list contains only one element, although there is still one point that has not been connected, it is an isolated point and cannot be connected, so the current round of the loop can also end here;

[0106] Exclude secondary connection points whose distance from the connection point exceeds a threshold, and calculate the angle θ constructed between the starting point, connection point, and secondary connection point; construct a score η by weighting the distance between connection points and secondary connection points and the angle θ between the starting point and secondary connection point; please refer to [link to relevant documentation]. Figure 8 In this embodiment of the invention, the angle θ is used to describe the curvature of the ternary connection group constructed by the feature points, with a value range of (0°, 180°). The larger the angle value, the gentler the curvature of the ternary connection group, and the easier it is to be selected as the final connection scheme. This is reflected by a weighted coefficient weighted to the score η. In this embodiment of the invention, the distance d is used to describe the distance between the connection point and the secondary connection point. The larger the distance d, the farther the secondary connection point is from the connection point, and the less likely it is to be selected as the final connection scheme. This is reflected by a weighted coefficient weighted to the score η. The angle θ and the distance d are weighted together to construct the score η. In this embodiment of the invention, the score η is used to score each "starting point-connection point-secondary connection point" ternary connection group in the current connection. The higher the score of the ternary connection group, the easier it is to be selected as the final connection scheme under the condition of only the information at the current moment. As shown in the formula in step 2-2-7 below. However, in fact, the final connection scheme of the current connection is not only based on the current information, but also on the existing historical information. Therefore, the connection probability P needs to be calculated later.

[0107] The connection probability P is constructed using a score η and the number of occurrences τ of the corresponding triplet connection group. It should be noted that the triplet connection group in this embodiment is a simplified representation of "starting point - connection point - secondary connection point." The number of occurrences τ in this embodiment represents historical information. Because tracking is a dynamic process, scene images are visually captured and geometrically separated at each moment. When performing geometric separation at the first moment, there is no historical information. However, in subsequent geometric separations, the number of occurrences of the corresponding triplet in history is considered in addition to the score η, prioritizing the connection with the highest historical occurrence. The final connection probability P is obtained through the calculation of the score η and the historical connection count τ, and the triplet connection group with the highest probability is selected as the final connection scheme.

[0108] The triplet with the highest connection probability P is selected as the connection scheme, and its occurrence count is incremented by 1. In this embodiment of the invention, the probability of each relevant triplet selected at this moment is calculated by combining the score η of the triplet at this moment with the historical occurrence count τ of that triplet. The triplet with the highest probability is ultimately selected as the final triplet for this moment, excluding other triplet selections. Increasing its occurrence count by 1 indicates that the connection result is recorded in the historical information for future use.

[0109] Replace the connection point with the new starting point, replace the selected secondary connection point with the new connection point, use the randomly selected point used for the forward line search connection as the connection point, and use the other endpoint of the starting edge of the forward line search as the starting point. This line search is called the reverse line search.

[0110] In this round of searching for connections along the line, elements that appear in the ternary connection group are denoted as feature points of the i-th linear deformable body that has been separated, and list D is constructed. i ;

[0111] Continue until list D is empty or contains only one element. Then, add the deleted element from list C to list D and restart the search for connection along the line of another linear transformation.

[0112] In practical applications, please refer to Figure 4 , Figure 5 , Figure 6 , Figure 7 and Figure 8 Step two of this embodiment specifically includes: using geometric methods to separate multiple linear deformable bodies, separating the elements in list D into num lists, denoted as D1, D2, ..., D... num num represents the number of DLOs in the scene.

[0113] Step 2-1: Identify the intersections of DLOs in list D and construct them as list Cr.

[0114] Step 2-1-1: Construct a KD-tree from all elements in list D, and query each element d. i The distances of a adjacent elements from element d are respectively i distance d i1 d i2 , ...,d ia (a is a manually set constant)

[0115] Step 2-1-2: Find the densest points in list D: sort each element d i1 d i2 , ...,d ia Add the values ​​together, extract the m smallest elements, and construct a list C. (m is a manually set constant.)

[0116] C = (c1, c2, ..., c m )=((u1,v1),(u2,v2),…(u m ,v m ))

[0117] Step 2-1-3: Use the k-means clustering method, with k init Cluster list C with the initial number of cluster cores. After obtaining the cluster cores, determine the distance between the cluster cores. If the minimum distance is less than thd... cross Then k init -x represents the number of clustering cores. Re-cluster list C (where x is the number of times it has been clustered) until the minimum distance is greater than or equal to thd. cross or k init The result is 1. The resulting cluster centers are the intersections of the DLO (Distributed Alternating Loop). These are then used to construct a list Cr, and each element in list C is separated according to its corresponding cluster center: (k init ,thd cross (For manually setting constants)

[0118]

[0119] Step 2-2: Randomly select an element in list D and start searching for connections along the line, separating the feature point elements belonging to the i-th DLO, and constructing them into list D. i .

[0120] Step 2-2-1: Randomly select an element d from list D. i This serves as the starting point for the search along the line, and this search along the line is recorded as a forward search along the line.

[0121] Step 2-2-2: Determine element d i Does element d belong to list C? If it does, use element d. iThe corresponding cluster center Cr j Replace element d with coordinates i And remove from list D the one with Cr j For each element in the list C of cluster centers, assign a value thd to it. a If it does not belong, use element d. i Use the coordinates of the starting point for the search along the line, and delete element d from list D. i And assign a value thd to the threshold thd. b (thd) a ,thd b To manually set constants, generally thd a >thd b )

[0122] Step 2-2-3: Based on the starting point coordinates, find the nearest adjacent element d in list D using a KD-tree. k If the distance of the element from the starting point is less than thd, then the element is marked as a connection point, and the edge from the starting point to this connection point is the starting edge; otherwise, go to step 2-2-12.

[0123] Step 2-2-4: Determine the connection point d k Does element d belong to list C? If it does, use element d. k The corresponding cluster center Cr m Replace element d with coordinates k And remove from list D the one with Cr m For each element in the list C of cluster centers, assign a value thd to it. a If it does not belong, use element d. k Use the coordinates of the connection points along the line, and delete element d from list D. k And assign a value thd to the threshold thd. b .

[0124] Step 2-2-5: Based on the coordinates of the join point, find the b nearest adjacent elements in list D using a KD-tree. These b adjacent elements are called secondary join points. (b is a manually set constant)

[0125] Step 2-2-6: Exclude secondary connection points whose distance from the connection point exceeds the threshold thd, and calculate the angle θ constructed by the starting point, connection point, and secondary connection point (calculate angles less than or equal to 180 degrees). If the distance of all secondary connection points from the connection point exceeds thd, proceed to step 2-2-11.

[0126] Step 2-2-7: Construct a score η by weighting the distance l between the connection point and the secondary connection point, and the angle θ between the starting point, the connection point, and the secondary connection point. The formula is:

[0127] η=-A×l+B×θ

[0128] A represents the distance weighting coefficient, which is a manually set constant; B represents the angle weighting coefficient, which is a manually set constant.

[0129] Step 2-2-8: Construct the connection probability P using the score η and the number of occurrences τ of the corresponding triplet connection group, with the formula:

[0130]

[0131] P ijk This indicates the probability that the current connection is a ternary connection group: starting point i - connection point j - secondary connection point k; α represents the weight of the number of times the ternary connection group has appeared, which is a manually set constant; β represents the weight of the score, which is a manually set constant.

[0132] Step 2-2-9: Select the connection probability P ijk The highest-ranking ternary connection group is selected as the connection scheme, and its occurrence count is incremented by 1.

[0133] Step 2-2-10: Replace the connection point with the new starting point, replace the selected secondary connection point with the new connection point, and go to step 2-2-4.

[0134] Step 2-2-11: Using the randomly selected point used for the forward line search connection as the connection point, and the other endpoint of the starting edge of the forward line search as the starting point, denote this line search as the reverse line search. Determine whether the reverse line search has been completed. If it has been completed, proceed to step 2-2-12. Otherwise, proceed to step 2-2-5.

[0135] Step 2-2-12: Record the elements that appear in the triplet connection group during the current search along the line as the feature points of the i-th DLO that have been separated, and construct a list D. i .

[0136] Step 2-2-13: Determine if list D is empty or contains only one element. If it is empty or contains only one element, proceed to step 2-2-15. Otherwise, proceed to step 2-2-14.

[0137] Step 2-2-14: Add the deleted elements belonging to list C to list D, and restart the line lookup connection for another DLO. Go to step 2-2-1.

[0138] Step 2-2-15: Perform a loss operation on the number of occurrences of the ternary connection group to provide a reference for the next dynamic iteration separation. The loss formula is:

[0139] τ(t+1)=ρ·τ(t)

[0140] Where τ(t+1) represents the number of times the ternary connection group used for the next dynamic iteration separation has appeared; ρ represents the loss coefficient, which is a manually set constant; and τ(t) represents the number of times the ternary connection group used for the current dynamic iteration separation has appeared.

[0141] Step 3: The robot autonomously manipulates the multilinear deformable body by combining the feature points in the n lists.

[0142] The robot in this embodiment of the invention, based on the objective of autonomously manipulating a linear deformable body, combines the feature point separation results D1, D2, ..., D of the linear deformable body. num Manipulate multilinear deformable bodies.

[0143] Step three above describes the robot's autonomous manipulation of the multilinear deformable body by combining the feature points from the n lists. Specifically:

[0144] Input the feature point information of the linear deformable body and the position and orientation information of the linear deformable body as desired by the user;

[0145] The robot autonomously selects a suitable grasping position and performs actions based on feature point information, the user's desired position information, and posture information, and then executes the operation steps.

[0146] Output the position and attitude information of the multilinear deformable body.

[0147] Step 4: Determine if the robot has completed its autonomous operation of the multilinear deformable body. If successful, the task ends; otherwise, proceed to Step 1. For details, please refer to [link to Step 4]. Figure 9 and Figure 10 Step four includes:

[0148] Input the position and attitude information of the linear deformable body as desired by the user, as well as the position and attitude information of the multilinear deformable body output in step three;

[0149] The user's desired location information is compared with the attitude information, and the location information output in step three is compared with the attitude information to obtain the comparison result; the comparison process in this embodiment of the invention includes: extracting using the SPR method. Figure 9 The feature points of the linear deformable body in the two figures will be Figure 10 The feature points in the graph are plotted on the same graph, and calculations are performed on that graph. Figure 9 The Euclidean distances of each feature point in the two figures are used to determine the completion of the robot's operation of the linear deformable body. When the sum of all Euclidean distances is below a threshold, the robot's operation of the linear deformable body is considered to be complete; otherwise, it is not yet complete.

[0150] Based on the comparison results, it is determined whether the robot's operation of the linear deformable body process has been completed, and based on whether it has been completed, it is selected to return to step one or end.

[0151] Furthermore, an embodiment of the present invention provides a method for tracking multilinear deformable bodies (hereinafter referred to as DLOs) in which the method is applied to... Figure 2 Examples include multilinear deformable objects, cameras, light sources, robots, and processors, see reference. Figure 2 As shown.

[0152] Figure 3 This is a flowchart illustrating the overall process of a multilinear deformable body tracking method based on structure-preserving registration and iterative geometric separation according to an embodiment of the present invention. In this embodiment, the first step is to extract and track the global features of the linear deformable body, including camera acquisition and extraction of scenes containing multiple DLOs, extraction and tracking of global feature points of the multiple DLOs, and constructing a list D of the global feature points of the multilinear deformable body. In this application example, the next step is to separate the feature points of multiple DLOs using geometric methods, dividing the feature points in list D into num lists, denoted as D1, D2, ..., D... num num represents the number of DLOs in the scene. After completion, the robot can perform relevant operations for multiple DLOs based on feature points and their respective classifications. After completing one operation, the robot determines whether the task is complete. If completed, the system terminates; otherwise, the above steps are repeated until the robot operation is complete. In this embodiment of the invention, the robot's autonomous operation process is embedded. As the robot autonomously operates on multiple linear deformable bodies, this embodiment can simultaneously perform separation and tracking of the multiple linear deformable bodies. That is, steps 1 and 2 of this embodiment are embedded into the robot's autonomous operation of linear deformable bodies (steps 3 and 4), enabling mutual separation and tracking while the robot autonomously operates on multiple DLOs, iteratively guiding the robot's autonomous operation.

[0153] exist Figure 3 In the global feature extraction and tracking shown, a camera is used to capture arbitrarily intersecting but independent multi-DLO scenes to obtain the original image I, which is then converted into a grayscale image I. gray .

[0154] The original image I is represented as a three-dimensional matrix of size H*W*3, where H represents the image height, W represents the image width, and 3 indicates that each pixel is divided into three channels: red, green, and blue (R, G, B). The values ​​of each element in the three-dimensional matrix of the original image I represent the brightness value of the corresponding channel at the corresponding location, ranging from 0 to 255, where 0 represents the darkest and 255 represents the brightest.

[0155] Grayscale image I grayThe image is represented as a two-dimensional matrix H*W, where H represents the image height and W represents the image width. Each element in the grayscale image's two-dimensional matrix represents the brightness value at the corresponding location, ranging from 0 to 255, where 0 represents the darkest and 255 represents the brightest. The values ​​of each element in the grayscale image are calculated from the three channel values ​​of the corresponding location in the original image, using the following formula:

[0156] Gray = 0.299R + 0.587G + 0.114B

[0157] exist Figure 3 In the global feature extraction and tracking shown, a color threshold-based mask is used for grayscale image I. gray The background and deformable body are segmented to obtain a list of the positions of all DLO pixels. def .

[0158]

[0159] in, The operation - Hadamard product - represents the multiplication of corresponding values ​​in two matrices of the same size; I def This is an image containing only the deformable objects after masking and segmentation; I gray M is a grayscale image; M is a color threshold-based mask.

[0160] The method for constructing the color threshold-based mask M according to the embodiments of the present invention is as follows:

[0161] Based on the user-defined threshold thd col When the value of the grayscale image matrix at a certain position is greater than or equal to the threshold, the corresponding position of the mask is set to 1; when the value of the grayscale image matrix at a certain position is less than the threshold, the corresponding position of the mask is set to 0. The formula is as follows:

[0162]

[0163] exist Figure 3 In the global feature extraction and tracking shown, the Structure Preserving Registration (SPR) method is used to extract and track list I. def Find all feature points in the DLO and construct a list D with the coordinates of the feature points (n is the number of feature points).

[0164] D = (d1, d2, ..., d n )=((u1,v1),(u2,v2),…(u n ,v n ))

[0165] exist Figure 3In the initial global feature separation shown, the first step is to extract the intersection points of multiple DLOs, constructing a candidate intersection point list Cr and an intersection point list C. This provides the necessary basic information for subsequent connection point searching along the line, such as... Figure 4 As shown, first, a KD-tree is constructed from all elements in list D, and then each element d is queried. i The distances of a adjacent elements from element d are respectively i distance d i1 d i2 , ...,d ia (a is a manually set constant). Next, identify the densest points in list D: [The text abruptly ends here, likely due to an incomplete sentence or a formatting error.] i1 d i2 , ...,d ia The values ​​are added together, and the m smallest elements are selected to form a list C (where m is a manually set constant). Finally, the k-means clustering method is used to cluster the elements, with k... init Cluster list C with the initial number of cluster cores. After obtaining the cluster cores, determine the distance between the cluster cores. If the minimum distance is less than thd... cross Then k init -x represents the number of clustering cores. Re-cluster list C (where x is the number of times it has been clustered) until the minimum distance is greater than or equal to thd. cross or k init The result is a cluster center of 1, which is the intersection of the DLO. This intersection is then used to construct a list Cr, and each element in list C is separated according to its cluster center (k...). init ,thd cross (For manually setting constants).

[0166] exist Figure 3 In the initial separation of global features shown, the global feature points are then separated into multiple feature point lists using a geometric method that finds connection points along the lines, as follows: Figure 5 As shown.

[0167] 1) First, randomly select an element d from list D. i This serves as the starting point for the search along the line, and this search along the line is recorded as a forward search along the line.

[0168] 2) Next, the starting point is processed by combining the cross-point backup list Cr and the cross-point list C. Based on the processed starting point coordinates, the nearest adjacent element d to this coordinate is found in list D using a KD-tree. k If the distance from the element to the starting point is less than thd, then the element is marked as a connection point, and the edge from the starting point to this connection point is the starting edge; otherwise, the current round of connection ends, and the elements connected in this round are used to construct a list D. i ,like Figure 6 As shown.

[0169] 3) Combine the cross-point backup list Cr and the cross-point list C to process the connection points. Based on the processed connection point coordinates, find the b nearest adjacent elements in list D using a KD-tree. These b adjacent elements are called secondary connection points (b is a manually set constant). Figure 7 As shown.

[0170] 4) Exclude secondary connection points whose distance from the initial connection point exceeds the threshold thd. If all secondary connection points are more than thd away from the initial connection point, consider the secondary connection point empty, swap the initial and initial connection points, and begin searching for connections along the line in reverse. Otherwise, calculate the angle θ formed by the initial point, connection point, and secondary connection point (calculate angles less than or equal to 180 degrees). Figure 8 As shown.

[0171] 5) Construct a score η by weighting the distance l between the connection point and the secondary connection point, and the angle θ between the starting point, the connection point, and the secondary connection point. The formula is:

[0172] η=-A×l+B×θ

[0173] A - Distance weighting coefficient, manually set as a constant; B - Angle weighting coefficient, manually set as a constant.

[0174] 6) Construct the connection probability P using the score η and the number of occurrences τ of the corresponding triplet connection group, with the formula:

[0175]

[0176] P ijk - This connection is a ternary connection group: the probability of starting point i, connection point j, and secondary connection point k; α - the weight of the number of times the ternary connection group has appeared, which is a manually set constant; β - the weight of the score, which is a manually set constant.

[0177] 7) Choose the connection probability P ijk The highest-ranking ternary link is selected as the linking scheme, and its occurrence count is incremented by 1. The selected link is replaced with the new starting point, and the selected secondary link is replaced with the new link. Proceed to step 3).

[0178] 8) The reverse search process is similar to the forward search process. If the reverse search is not completed, proceed to step 3); otherwise, the current connection round ends, and the elements connected in this round are used to construct a list D. i ,and

[0179] 9) Perform a loss operation on the number of occurrences of ternary connection groups to provide a reference for the next dynamic iteration separation. The loss formula is:

[0180] τ(t+1)=ρ·τ(t)

[0181] τ(t+1) - the number of times the ternary connection group used for separation in the next dynamic iteration has appeared; ρ - loss coefficient, a manually set constant; τ(t) - the number of times the ternary connection group used for separation in this dynamic iteration has appeared;

[0182] 10) Determine if the separation is complete. If complete, end the global feature separation process; otherwise, continue to randomly select from the remaining feature points and begin a new round of forward line search.

[0183] exist Figure 2 In the robot operation shown, the robot, based on the objective of autonomous operation DLO, combines feature points and preliminary separation results D1, D2, ..., D... num Perform linear deformation operations and obtain new multi-DLO states.

[0184] exist Figure 3 The task completion check shown determines whether the robot has successfully completed its autonomous manipulation of the linear deformable body. If successful, the task ends. Otherwise, it returns to global feature extraction and tracking.

[0185] Please see Figure 11 The multilinear deformable body tracking method of this invention is embedded in the process of robot autonomous operation of linear deformable bodies. Figure 11 The left-hand graph shows the states of a series of linear deformable bodies obtained through autonomous robot operation. Figure 11 The right-hand graph shows the separation and tracking of linear deformable bodies, with each result used to guide the robot's autonomous operation in the next moment. This embodiment of the invention combines iterative geometric separation with single-linear deformable body tracking based on the SPR method. While tracking multiple linear deformable bodies as a whole, it simultaneously separates each linear deformable body in real time, achieving separate tracking of each DLO (Dependency Locator).

[0186] The detailed descriptions listed above are merely specific descriptions of feasible embodiments of the present invention, and are not intended to limit the scope of protection of the present invention. All equivalent embodiments or modifications made without departing from the spirit of the present invention should be included within the scope of protection of the present invention.

[0187] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0188] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A method for tracking multilinear deformable bodies based on structure-preserving registration and iterative geometric separation, characterized in that, include: Step 1: Extract and track feature points of the multilinear deformable body using a structure-preserving registration model, and construct a feature list based on the feature points; Step 2: Separate n linear deformable bodies using geometric methods. Based on the linear deformable bodies, separate the feature points in the feature list into n lists, where n is the number of linear deformable bodies; specifically including: Extract the cross candidate points of the multilinear deformable body and construct a cross candidate point list; extract the cross points based on the cross candidate points and construct the cross point list. Based on the feature list and the intersection point list from step one, feature points belonging to the i-th multilinear deformable body are separated to construct list D. i This results in n lists; The specific process of extracting candidate intersection points of a multilinear deformable body and constructing a candidate intersection point list, and extracting intersection points based on the candidate intersection points and constructing the intersection point list, includes: Construct a KD-tree from all elements in the feature list described in step one, and query each element d in the KD-tree. i The distances of a adjacent elements from element d are respectively i distance d i1 d i2 , ..., d ia ; Find the densest points in the feature list, and assign each element d... i d i1 d i2 , ..., d ia The sum of each element is obtained by adding them together. The m elements with the smallest sum are then used to construct a list of cross-crossing candidate points C, where m represents the number of cross-crossing candidate points. The k-means clustering method is used to cluster the cross-point candidate list C to obtain cluster centers. The cluster centers are then used to construct the cross-point list Cr, and each element in the cross-point candidate list C is separated according to its respective cluster center. Step 3: The robot autonomously manipulates the multilinear deformable body by combining the feature points from the n lists; Step 4: Determine whether the robot has completed its autonomous operation of the multilinear deformable body. If it has, the task ends; otherwise, proceed to Step 1.

2. The multilinear deformable body tracking method based on structure-preserving registration and iterative geometric separation according to claim 1, characterized in that, Step one includes: An original image is obtained by acquiring an image of a scene containing multiple linear deformable bodies, and the original image is then processed to obtain a grayscale image. The multilinear deformable body and the background in the grayscale image are segmented to obtain a list of the positions of each pixel of the multilinear deformable body. Feature points are obtained by extracting and tracking the location list using a structure-preserving registration model, and a feature list is constructed based on the coordinates of the feature points.

3. The method for tracking multilinear deformable bodies based on structure-preserving registration and iterative geometric separation according to claim 2, characterized in that, Step one specifically involves: A scene of arbitrarily intersecting but independent multilinear deformable bodies is captured using a camera to obtain an original image I, which is then converted into a grayscale image I. gray ; The grayscale image I is processed using a color threshold-based mask. gray The multilinear deformable body is segmented from the background to obtain a list of the positions of all multilinear deformable body pixels. def , is represented as: in, This represents the Hadamard product operation, which multiplies the corresponding values ​​of two matrices of the same size. This represents a mask based on a color threshold. The location list I is extracted and tracked using a structure-preserving registration model. def All feature points of a multilinear deformable body are used to construct a feature list D based on the coordinates of the feature points. Where D represents the feature list, d represents the feature point in the list, and (u, v) represents the x and y coordinates of the corresponding position of the feature point.

4. The multilinear deformable body tracking method based on structure-preserving registration and iterative geometric separation according to claim 1, characterized in that, The feature points belonging to the i-th multilinear deformable body are separated from the feature list and the intersection point list in step one to construct list D. i ,include: Randomly select a feature point d from the feature list described in step one. i Starting from the point, find the connection point d along the line in a forward direction. k ; Determine the connection point d k If the distance from the starting point is less than a threshold, then determine the connection point d. k If the number of secondary connection points exceeds a threshold, then a list D is constructed using the feature points found along the current line. i； Based on connection point d k Find secondary connection points, or after swapping the initial and connection points used initially, search for secondary connection points along the line in reverse. If a secondary connection point exists, select a suitable secondary connection point, replace the initial point with the connection point, and replace the connection point with the secondary connection point. If no secondary connection point exists, separate the feature points found along the line in this round of searching for connections, until a list D of feature points belonging to the i-th multilinear deformable body is constructed from the feature list and the intersection point list. i .

5. The multilinear deformable body tracking method based on structure-preserving registration and iterative geometric separation according to claim 4, characterized in that, Randomly select a feature point d from the feature list described in step one. i Starting from the point, find the connection point d along the line in a forward direction. k Specifically: Randomly select a feature point d from the feature list described in step one. i As the starting point of the search along the line, this search along the line is denoted as a forward search along the line; Determine feature point d i If a feature point belongs to the list of crossover candidate points, then feature point d is used. i The corresponding cluster center Cr j Replace feature point d with coordinates i And remove Cr from the feature list j The cluster centers are all elements in the list C; if an element does not belong, the cluster center is represented by its feature point d. i As the coordinates of the starting point for searching along the line, delete feature point d from the feature list. i Find feature point d in the feature list based on the starting point coordinates. k .

6. The multilinear deformable body tracking method based on structure-preserving registration and iterative geometric separation according to claim 4, characterized in that, Determine the connection point d k If the distance from the starting point is less than a threshold, then determine the connection point d. k If the number of secondary connection points exceeds a threshold, then a list D is constructed using the feature points found along the current line. i According to the connection point d k Find secondary connection points, or after swapping the initial and connection points used initially, search for secondary connection points along the line in reverse. If a secondary connection point exists, select a suitable secondary connection point, replace the initial point with the connection point, and replace the connection point with the secondary connection point. If no secondary connection point exists, separate the feature points found along the line in this round of searching for connections, until a list D of feature points belonging to the i-th multilinear deformable body is constructed from the feature list and the intersection point list. i ; include: Based on feature point d k If the distance from the starting point is less than a threshold, then the adjacent feature points d are considered. k Let the starting point be the connection point, and determine the edge from the starting point to the connection point as the starting edge; Determine adjacent feature points d k If a point belongs to the list of crossover candidate points, then use the adjacent feature point d. k The corresponding cluster center Cr m Replace feature point d with coordinates k And remove Cr from the feature list m The cluster centers are all elements in the list C; if an element does not belong, the cluster center is represented by its feature point d. k As the coordinates of the connection points searched along the line, feature point d is deleted from the feature list. k Based on the coordinates of the connection point, find the b nearest adjacent elements in the feature list as secondary connection points; Exclude secondary connection points whose distance from the connection point exceeds a threshold, and calculate the angle θ constructed between the starting point, connection point, and secondary connection point; construct a score η by weighting the distance between connection points and secondary connection points and the angle θ between the starting point and connection point; The connection probability P is constructed by the score η and the number of occurrences τ of the corresponding triple connection group. Select the triplet with the highest connection probability P as the connection scheme, and increment its occurrence count by 1. Replace the connection point with the new starting point, replace the selected secondary connection point with the new connection point, use the randomly selected point used for the forward line search connection as the connection point, and use the other endpoint of the starting edge of the forward line search as the starting point. This line search is called the reverse line search. In this round of searching for connections along the line, elements that appear in the ternary connection group are denoted as feature points of the i-th linear deformable body that has been separated, and list D is constructed. i ; Continue until list D is empty or contains only one element. Then, add the deleted element from list C to list D and restart the search for connection along the line of another linear transformation.

7. The multilinear deformable body tracking method based on structure-preserving registration and iterative geometric separation according to claim 1, characterized in that, Step three, where the robot autonomously manipulates the multilinear deformable body by combining the feature points from the n lists, specifically involves: Input the feature point information of the linear deformable body and the position and orientation information of the linear deformable body as desired by the user; The robot autonomously selects a suitable grasping position and performs actions based on feature point information, the user's desired position information, and posture information, and then executes the operation steps. Output the position and attitude information of the multilinear deformable body.

8. The multilinear deformable body tracking method based on structure-preserving registration and iterative geometric separation according to claim 1, characterized in that, Step four, determining whether the robot has completed its autonomous operation of the multilinear deformable body, includes: Input the position and attitude information of the linear deformable body as desired by the user, as well as the position and attitude information of the multilinear deformable body output in step three; The user's desired location information is compared with the attitude information, and the location information output in step three is compared with the attitude information to obtain the comparison result. Based on the comparison results, it is determined whether the robot's operation of the linear deformable body process has been completed, and based on whether it has been completed, it is selected to return to step one or end.

Images