Guide wire contact state sensing method based on visual tactile sensor

Abstract

The invention relates to the technical field of guide wire state sensing, in particular to a guide wire contact state sensing method based on a visual tactile sensor, which is characterized in that the visual tactile sensor is used as equivalent replacement of fingers of an operator, the operator operates a robot through information fed back by the visual tactile sensor when controlling the robot to work, and the visual tactile sensor is used for sensing the contact state of the guide wire during intervention. The guide wire is in direct contact with the flexible contact module of the sensor, when the guide wire is in contact with the blood vessel wall to generate stress, the stress is transmitted to the visual touch sensor along the guide wire, the elastic body is slightly deformed, the stress change trend of the guide wire can be sensed through the displacement change of the mark point in the plane, and visual feedback is carried out. The real-time stress image and the sudden change stress image of the guide wire are generated in combination with a gray value mapping mode, and real-time and easily understood stress feedback information is provided for an operator, so that the operator can know the overall stress trend of the guide wire in time and can also accurately perceive local abnormal signals.

Description

A method for sensing the contact state of a guidewire based on a visual-tactile sensor Technical Field

[0001] This invention relates to the field of guidewire state sensing technology, and specifically to a method for sensing guidewire contact state based on a visual-tactile sensor. Background Technology

[0002] Interventional vascular surgery, as a minimally invasive treatment method, is widely used in the diagnosis and treatment of various vascular diseases. During the procedure, the surgeon must operate precisely to ensure the guidewire safely reaches the target location within the patient's blood vessel. During the operation, contact forces are generated between the guidewire and the vessel wall due to factors such as friction, contact pressure, geometric bending, and resistance. The magnitude of the contact force directly affects the stability of instrument delivery and the safety of the surgery. Insufficient contact force may lead to guidewire deviation or failure to pass through the lesion effectively; excessive contact force may cause compression, abrasion, or even perforation of the vessel wall. Therefore, real-time sensing of changes in the contact force between the guidewire and the blood vessel is of great significance for improving the safety and precision control of interventional procedures.

[0003] Existing force sensing devices are often bulky, have limited sampling rates, and lack sensitivity to small forces, making it difficult to capture the subtle changes in contact forces commonly found within blood vessel cavities. Some studies have used wearable devices with integrated force visual-tactile sensors, but these devices suffer from being bulky and having large measurement errors. Coaxial force-torque visual-tactile sensors, on the other hand, are unable to accurately capture subtle contact forces. As a result, there is still a lack of effective means to perceive the guidewire contact status in real time and with precision during vascular intervention. Therefore, a guidewire contact status perception method based on visual-tactile sensors is proposed. Summary of the Invention

[0004] To address the technical problems existing in the prior art, the present invention provides the following technical solution: a guide wire contact state sensing method based on a visual-tactile sensor, the guide wire contact state sensing method comprising the following steps:

[0005] S1, extract deformed point clouds from the complete tactile point cloud caused by guide wire contact or affected by local elastic conduction, and remove noise points;

[0006] S2, by extracting the main axis of the denoised deformed point cloud, the marker point closest to the main axis is defined as the sensing contact point, thereby realizing rapid positioning of the guide wire contact area;

[0007] S3, extract the sensing contact point cloud, analyze the changing characteristics of the sensing contact point cloud in adjacent frames, and identify the contact state between the guide wire and the visual tactile sensor;

[0008] S4 employs a dual-channel visualization strategy. When the guidewire and the visual-tactile sensor maintain reliable contact (i.e., there is no relative slippage between them), it detects the minute tangential displacement of the visual-tactile sensor marker array in the x–y plane caused by the force on the guidewire. The displacement is then mapped in grayscale to construct a real-time force visualization image and a sudden change response visualization image of the guidewire, respectively, thereby achieving real-time and accurate feedback on the force trend of the guidewire.

[0009] Preferably, step S1 specifically includes the following steps:

[0010] S11, Deformed point cloud obtained by screening: When the guide wire comes into contact with the visual-tactile sensor, a local depression will be formed on the surface of the elastic body of the visual-tactile sensor. This deformation is mainly manifested as displacement along the z-axis in the point cloud. The z-axis displacement is used as the initial deformation criterion to screen out the deformed point cloud.

[0011] S12, Filtering out noise points: The isolated point filtering method based on local neighborhood statistics is used to filter the deformed point cloud.

[0012] Preferably, in step S11, during the filtering process, the midpoint of the reference frame is set. The coordinates in the z-axis direction are The point corresponding to the current frame The coordinates in the z-axis direction are Then its z-axis displacement can be expressed as:

[0013] ;

[0014] When the z-axis displacement of a certain point satisfies If the point is found to be deformed, it indicates that the point has undergone deformation and is identified as a deformation point caused by the contact of the guide wire. The point set obtained by this screening constitutes a preliminary deformed point cloud.

[0015] Preferably, in step S12, during the removal process, each point in the deformed point cloud is constructed. A local neighborhood window centered at the center is defined, with each local neighborhood window having a side length of 2r+1, where r is the neighborhood radius. All local neighborhood windows containing values ​​whose z-axis displacement is greater than a threshold are counted. The neighboring points are the valid neighboring points;

[0016] Calculation points The number of valid neighbors within a local neighborhood is used to determine the validity of a point. When the number of valid neighbor points exceeds half of the total number of points in the neighborhood, the point is considered valid. The region in question exhibits continuous deformation characteristics, and the points... The points are retained in the deformed point cloud; conversely, if the number of effective neighboring points is less than half, the points are removed. Isolated noise points are identified and removed to obtain the denoised deformed point cloud.

[0017] Preferably, step S2 specifically includes the following steps:

[0018] S21, the physical contact between the guidewire and the visual-tactile sensor can be approximated as a straight line. Based on the PCA method, the deformed point cloud is analyzed, and the eigenvector corresponding to the largest eigenvalue is selected as the principal axis of the deformed point cloud.

[0019] S22, Calculate the perpendicular distance from each point in the deformed point cloud to the principal axis. To further determine the contact position of the guide wire on the surface of the visual-tactile sensor, the calculation formula is as follows:

[0020] ;

[0021] in, The center point of the deformed point cloud is the average coordinate of all points. The eigenvector corresponding to the largest eigenvalue;

[0022] When a point in the deformable point cloud satisfies When this point is defined as the guide wire contact point sensed by the visual-tactile sensor, all points that meet the conditions constitute a sensing contact point cloud.

[0023] S23, directly assign a specific color to the selected sensing contact point cloud to distinguish the sensing contact point cloud from other point clouds, so that the contact position of the guide wire on the surface of the visual tactile sensor can be intuitively presented in the point cloud, and the positioning of the guide wire contact position can be realized.

[0024] Preferably, step S3 specifically includes the following steps:

[0025] S31, using the overlap rate R of adjacent frame contact areas as a criterion, determines the state between the guidewire and the visual-tactile sensor, specifically:

[0026] ;

[0027] in, and These are the sets of perceptual contact point cloud indices for the current frame and the previous frame, respectively. The cardinality of the intersection of the point cloud indices of the contact regions in two adjacent frames;

[0028] The contact status shall be determined according to the following rules:

[0029] ;

[0030] in, The minimum threshold number of sensing contact points. For contact consistency threshold;

[0031] When the number of perceived contact point clouds is less than a threshold When the current point cloud size is insufficient to stably characterize the effective contact behavior between the guide wire and the visual-tactile sensor, the system directly determines it as a non-contact state.

[0032] When the number of perceived contact point clouds is greater than or equal to the threshold When, if the overlap rate R < If the guidewire is in a rolling state with the visual-tactile sensor, it is determined that the contact is in a non-rolling state; otherwise, it is determined that the contact is in a non-rolling state.

[0033] S32, in order to avoid the process of the guide wire and the visual tactile sensor just making contact and gradually entering a stable contact state being misjudged as a rolling state, a 3-frame tolerance mechanism is adopted: within the 3-frame stage, regardless of whether the contact area changes, the system determines that it is a contact without rolling state.

[0034] Preferably, step S4 specifically includes the following steps:

[0035] S41, when the guide wire makes reliable contact with the visual tactile sensor, the guide wire is subjected to external force, which will cause the array of marker points on the surface of the visual tactile sensor to produce a small tangential displacement in the x–y plane. The change of tangential displacement of the marker points is detected to obtain the relative displacement of each marker point in the tangential direction.

[0036] S42, Constructing a real-time force visualization image: A logarithmic mapping function is used to map the tangential displacement of each marker point in the marker array to a grayscale value, achieving an intuitive expression of visual information from changes in physical quantities. The logarithmic mapping function is:

[0037] ;

[0038] in, Let represent the tangential displacement amplitude of the marker point in the i-th row and j-th column of the marker point array. Tangential displacement The corresponding visual grayscale value, k is the enhancement coefficient, and A is the upper limit reference value of the displacement mapping;

[0039] S43, Construct a visualization of the mutation response: Establish a dynamically updated reference displacement baseline for each marker point. , The update method is an exponential moving average, and its update formula is:

[0040] ;

[0041] in, Forgetting factor, The initial value is determined by the displacement value when the visual-tactile sensor first detects a non-zero tangential displacement of the marker point;

[0042] get Then, calculate the tangential displacement of the current frame marker point relative to... The difference :

[0043] ;

[0044] when When the increase is large, it can be considered that there is an abnormal surge at this location, which may correspond to risks such as local blockage in front of the guidewire, increased friction, or contact with abnormal tissue;

[0045] right The same logarithmic mapping function as in step S42 is used to map the values ​​to grayscale to generate a mutation image.

[0046] Preferably, step S41 specifically includes the following steps:

[0047] S411, a reference point cloud is constructed using multi-frame statistics for marker point displacement calculation: when the system is in a stable state without external force, several frames of marker point cloud data are selected, the spatial coordinates of each marker point in the selected frames are statistically analyzed, and the median is taken as the reference coordinate of the marker point, thereby constructing a robust reference point cloud.

[0048] S412, after constructing the reference point cloud, the relative displacement of each marker point in the x, y, and z directions is obtained by calculating the coordinate differences between the current frame point cloud and the reference point cloud. The calculation formula is as follows:

[0049] ;

[0050] in, The coordinates of the marker point in the i-th row and j-th column of the current frame. For the coordinates of the marker point in the i-th row and j-th column of the reference point cloud, Let be the displacement of the marker point in row i and column j relative to the reference point cloud on the x, y, and z axes. If there is no point cloud data for the marker point in row i and column j in the current frame, then is recorded as . (0,0,0);

[0051] After obtaining the relative displacements of each marker point in the x, y, and z directions, the tangential displacement of the marker point in the x–y plane is then determined. for: .

[0052] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0053] 1. This invention uses a visual-tactile sensor as an equivalent replacement for the operator's fingers. When the operator controls the robot, the robot is operated by the information fed back by the visual-tactile sensor. During intervention, the guidewire directly contacts the flexible contact module of the sensor. When the guidewire contacts the blood vessel wall and generates force, the force is transmitted along the guidewire to the visual-tactile sensor, causing the elastic body to undergo slight deformation. The trend of the force change on the guidewire can be sensed by the displacement change of the marked point in the plane, and visual feedback can be provided.

[0054] 2. This invention achieves localized deformation of the surface of the visual-tactile sensor's elastomer when the guidewire comes into contact with it, and reconstructs a complete tactile point cloud of the sensor's elastomer. From this reconstructed point cloud, it extracts the set of points deformed by the guidewire contact or by local elastic conduction. Then, it analyzes the spatial characteristics of this deformed point cloud using PCA to determine the contact position, identify and judge the contact state, and visualize it. This enables precise and visual perception of the guidewire contact state, and allows for stable perception of guidewire contacts with a diameter not exceeding 0.2 mm, demonstrating high sensing sensitivity.

[0055] 3. This invention utilizes the fact that when the guide wire comes into physical contact with the elastomer, the deformation caused on the surface of the elastomer usually exhibits the characteristics of local continuity and patchy distribution in space, which effectively suppresses isolated noise points. Thus, deformed point clouds with good spatial coherence and structural consistency can be extracted from the complete tactile point cloud, providing a reliable data foundation for subsequent principal axis analysis based on geometric distribution characteristics.

[0056] 4. In this invention, by detecting the tangential displacement change of the marked point and mapping it to image brightness information, an intuitive and visual representation of the change in guidewire contact force is achieved. This provides the operator with real-time and easily understandable force feedback information. With the combined display of real-time and abrupt change images, the operator can not only understand the overall force trend of the guidewire in a timely manner, but also accurately perceive local abnormal signals. While maintaining the real-time performance of the system, the sensitivity and reliability of abnormality detection are significantly improved, which is of great significance for improving the safety of interventional surgery. It can generate a recognizable response to guidewire contact force changes on the order of approximately 0.02N, achieving effective perception of minute contact force changes.

[0057] 5. This invention introduces a visual-tactile sensor into the field of guidewire contact state perception. By utilizing its high spatial resolution perception capability for elastic body deformation and its visualization imaging characteristics, it achieves simultaneous acquisition of the contact position, contact state, and force changes of the guidewire on the surface of the visual-tactile sensor. Compared with existing guidewire sensing technologies that rely on single-point force sensing, strain sensing, or model-based indirect estimation methods, this invention overcomes the shortcomings of single-dimensional sensing information, lack of spatial distribution information, and difficulty in intuitively presenting the sensing results, significantly improving the richness and intuitiveness of guidewire contact perception. Attached Figure Description

[0058] Figure 1 is a schematic diagram of the overall flow of the guidewire contact state sensing method of the present invention;

[0059] Figure 2 is a complete tactile point cloud diagram of the guide wire contact visual-tactile sensor of the present invention;

[0060] Figure 3 is a schematic diagram of the wire positioning result of the present invention;

[0061] Figure 4 is a schematic diagram of an example of interactive state recognition according to the present invention;

[0062] Figure 5 is a visualization diagram of the sensing effect when different forces are applied to the guidewire according to the present invention;

[0063] Figure 6 is a schematic diagram showing the comparison of the visualization results of the guidewire of the present invention under different contact states after passing through the bending structure. Detailed Implementation

[0064] The present invention will be further described below with reference to the accompanying drawings and embodiments, which illustrate the above and other technical features and advantages of the present invention. However, the following embodiments are merely preferred embodiments of the present invention and are not exhaustive.

[0065] Example 1:

[0066] As shown in Figures 1-6, the present invention provides a guidewire contact state sensing method based on a visual-tactile sensor. The guidewire contact state sensing method includes the following steps:

[0067] S1, extract deformed point clouds from the complete tactile point cloud caused by guide wire contact or affected by local elastic conduction, and remove noise points;

[0068] S2, by extracting the main axis of the denoised deformed point cloud and defining the marker point closest to the main axis as the sensing contact point, the rapid positioning of the guide wire contact area is achieved;

[0069] S3, extract the sensing contact point cloud, analyze the changing characteristics of the sensing contact point cloud in adjacent frames, and identify the contact state between the guide wire and the visual tactile sensor;

[0070] S4 employs a dual-channel visualization strategy. When the guidewire and the visual-tactile sensor maintain reliable contact, it detects the minute tangential displacement of the visual-tactile sensor marker array in the x–y plane caused by the force applied to the guidewire. The displacement is then mapped in grayscale to construct real-time force visualization images and abrupt response visualization images of the guidewire, respectively, thereby achieving real-time and accurate feedback on the force trend of the guidewire.

[0071] In this embodiment, step S1 specifically includes the following steps:

[0072] S11, the deformed point cloud is obtained by screening: when the guide wire comes into contact with the visual tactile sensor, a local depression will be formed on the surface of the elastic body of the visual tactile sensor. This deformation is mainly manifested as displacement along the z-axis in the point cloud. Therefore, the z-axis displacement is used as the preliminary deformation criterion.

[0073] Specifically, let the midpoint of the reference frame be... The coordinates in the z-axis direction are The point corresponding to the current frame The coordinates in the z-axis direction are Then its z-axis displacement can be expressed as:

[0074] ;

[0075] When the z-axis displacement of a certain point satisfies (threshold) When the value is 0.3mm, it indicates that the point has deformed and is identified as a deformation point caused by the guide wire contact. The point set obtained by this screening constitutes a preliminary deformed point cloud, which provides a data basis for subsequent noise removal and PCA-based contact position localization.

[0076] S12, Filtering out noise points: The isolated point filtering method based on local neighborhood statistics is used to filter the deformed point cloud.

[0077] Specifically, it is constructed using each point in the deformable point cloud. A local neighborhood window is defined around the central point, with a side length of 2r + 1, where r is the neighborhood radius (r = 2, considering the distribution characteristics of point cloud density and noise). All points within the local neighborhood window whose z-axis displacement is greater than a threshold are counted. The neighboring points are the valid neighboring points;

[0078] Calculation points The number of valid neighbors within a local neighborhood is used to determine the validity of a point. When the number of valid neighbor points exceeds half of the total number of points in the neighborhood, the point is considered valid. The region in question exhibits continuous deformation characteristics, and the points... The points are retained in the deformed point cloud; conversely, if the number of effective neighboring points is less than half, the points are removed. Isolated noise points are identified and removed to obtain the denoised deformed point cloud.

[0079] In this embodiment, step S2 specifically includes the following steps:

[0080] S21. Although the guide wire material has a certain degree of flexibility, its overall stiffness is high. When it comes into contact with the surface of the visual-touch sensor, the degree of bending is limited. At the same time, the surface of the elastomer of the visual-touch sensor is convex, and the area involved in the contact between the guide wire and the visual-touch sensor is small. Therefore, the physical contact between the guide wire and the visual-touch sensor can be approximated as a straight line, and the principal axis of the deformed point cloud can be extracted based on PCA (Principal Component Analysis).

[0081] S22, Calculate the perpendicular distance from each point in the deformed point cloud to the principal axis. To further determine the contact position of the guide wire on the surface of the visual-tactile sensor, the calculation formula is as follows:

[0082] ;

[0083] When a point in the deformable point cloud satisfies When the threshold is 0.5mm, the point is defined as the guide wire contact point sensed by the visual tactile sensor, and all points that meet the condition constitute the sensing contact point cloud.

[0084] S23, directly assign a specific color to the selected sensing contact point cloud to distinguish the sensing contact point cloud from other point clouds, so that the contact position of the guide wire on the surface of the visual tactile sensor can be intuitively presented in the point cloud, and the positioning of the guide wire contact position can be realized.

[0085] In this embodiment, step S21 specifically includes the following steps:

[0086] S211, Suppose that the deformable point cloud dataset P has n points, and each point can be represented as... Let the point cloud dataset P be represented as an n×3 matrix, specifically:

[0087] ;

[0088] S212, Calculate the center point of point cloud P. :

[0089] ;

[0090] S213, decentralize the point cloud data, that is, subtract the center point from each point. The decentralized point cloud matrix is ​​obtained. , ;

[0091] S214, Calculate the covariance matrix of a 3D point cloud. :

[0092] ;

[0093] Where T is the transpose of the matrix;

[0094] For covariance matrix Performing eigenvalue decomposition yields: ,in, For the reason Eigenvalues The resulting diagonal matrix, Q, is a matrix composed of eigenvectors, specifically:

[0095] ;

[0096] ;

[0097] in, , Eigenvalues The corresponding feature vectors;

[0098] Obtain the three principal axis directions of point cloud P Select the largest eigenvalue Corresponding feature vector As the main axis of the deformable point cloud.

[0099] In this embodiment, step S3 specifically includes the following steps:

[0100] S31, using the overlap rate R of adjacent frame contact areas as a criterion, determines the state between the guidewire and the visual-tactile sensor, specifically:

[0101] ;

[0102] in, and These are the sets of perceptual contact point cloud indices for the current frame and the previous frame, respectively. The cardinality of the intersection of the point cloud indices of the contact regions in two adjacent frames;

[0103] The contact status shall be determined according to the following rules:

[0104] ;

[0105] in, To set the minimum threshold number of sensing contact points, take... It is 30. To determine the contact consistency threshold, take... It is 0.75;

[0106] When the number of perceived contact point clouds is less than a threshold When the current point cloud size is insufficient to stably characterize the effective contact behavior between the guide wire and the visual-tactile sensor, the system directly determines it as a non-contact state.

[0107] When the number of perceived contact point clouds is greater than or equal to the threshold When, if the overlap rate R < If the guidewire is in a rolling state with the visual-tactile sensor, it is determined that the contact is in a non-rolling state; otherwise, it is determined that the contact is in a non-rolling state.

[0108] S32. As the guidewire and visual-tactile sensor begin to make contact and gradually enter a stable contact process, the number of sensing contact points usually increases frame by frame. This process is easily misjudged as the guidewire rolling. To avoid the contact establishment process being misjudged as a rolling state, a 3-frame tolerance mechanism is adopted: within the 3-frame stage, regardless of whether the contact area changes, the system determines that the contact is in a non-rolling state, thereby effectively improving the stability and robustness of contact state recognition.

[0109] In this embodiment, step S4 specifically includes the following steps:

[0110] S41, when the guide wire makes reliable contact with the visual-tactile sensor, the guide wire is subjected to an external force, which causes a small tangential displacement of the array of marker points on the surface of the visual-tactile sensor in the x–y plane. The change in tangential displacement of the marker points is detected to obtain the relative displacement of each marker point in the tangential direction. Specifically, this includes the following steps:

[0111] S411 To avoid being affected by factors such as changes in illumination, detection errors, or local noise, a reference point cloud is constructed using multi-frame statistics. When the system is in a stable state without external force, the reference point cloud data of frames 111–115 are selected for reference point cloud computing. The spatial coordinates of each marker point in the selected 5 frames are statistically analyzed, and the median is taken as the reference coordinate of the marker point, thereby constructing a robust reference point cloud.

[0112] S412, after constructing the reference point cloud, the relative displacement of each marker point in the x, y, and z directions is obtained by calculating the coordinate differences between the current frame point cloud and the reference point cloud. The calculation formula is as follows:

[0113] ;

[0114] in, The coordinates of the marker point in the i-th row and j-th column of the current frame. For the coordinates of the marker point in the i-th row and j-th column of the reference point cloud, Let be the displacement of the marker point in row i and column j relative to the reference point cloud on the x, y, and z axes. If there is no point cloud data for the marker point in row i and column j in the current frame, then is recorded as . Given (0,0,0), after obtaining the relative displacements of each marker point in the x, y, and z directions, calculate the tangential displacement of the marker points in the x–y plane. :

[0115] ;

[0116] S42, Construct a real-time force visualization image (real-time image): The tangential displacement of each marker point in the marker array is mapped to a grayscale value using a logarithmic mapping function, realizing an intuitive expression of the change of physical quantity to visual information. The logarithmic mapping function is:

[0117] ;

[0118] in, Let represent the tangential displacement amplitude of the marker point in the i-th row and j-th column of the marker point array. Tangential displacement The corresponding visual grayscale value, k is the enhancement coefficient, and A is the upper limit reference value of the displacement mapping;

[0119] S43, Construct a visualization image of the mutation response (mutation image): Establish a dynamically updated reference displacement baseline for each marker point. , The update method is an exponential moving average, and its update formula is:

[0120] ;

[0121] in, The forgetting factor is set to 0.99. The initial value is determined by the displacement value when the visual-tactile sensor first detects a non-zero tangential displacement of the marker point;

[0122] get Then, calculate the tangential displacement of each marker point in the current frame relative to... The difference :

[0123] ;

[0124] when When the increase is large, it can be considered that there is an abnormal surge at this location, which may correspond to risks such as local blockage in front of the guidewire, increased friction, or contact with abnormal tissue;

[0125] right The same logarithmic mapping function as in step S42 is used to map the values ​​to grayscale to generate a mutation image.

[0126] Example 2:

[0127] To verify the effectiveness of the proposed guidewire contact position positioning method, industrial steel wire with material properties and mechanical stiffness similar to clinical interventional guidewires was selected as the experimental subject.

[0128] The main reason for choosing steel wire is its standardized specifications, which facilitates repeatable experiments under controllable variable conditions and helps to systematically evaluate the positioning ability and sensing limit of visual-tactile sensors.

[0129] During the experiment, a series of steel wire samples with diameters of 0.6 mm, 0.5 mm, 0.4 mm, 0.3 mm, 0.2 mm and 0.1 mm were selected and placed within the effective sensing area of ​​the visual-tactile sensor for testing.

[0130] To avoid large-scale deformation of the elastomer surface, uneven contact force, or introduction of additional noise caused by direct finger pressure, the experiment used a transparent acrylic plate as the force transmission medium. By applying vertical pressure to the surface of the acrylic plate, a stable and controllable contact state was formed between the steel wire and the surface of the visual-touch sensor to simulate the contact situation when the guide wire is clamped.

[0131] During the experiment, tactile point cloud data were collected for each wire diameter condition, and deformable point cloud extraction, neighborhood filtering for noise reduction, and PCA-based contact position localization analysis were performed in sequence. The contact position localization results of the guide wire under different wire diameter conditions are shown in Figure 3. The first row shows the real scene of the wire contacting the visual tactile sensor; the second row shows the visualization effect of the perceived tactile point cloud; and the third row shows the visualization of the localization results.

[0132] Experimental results show that the visual-tactile sensor can achieve stable contact position positioning for steel wires with a diameter of not less than 0.2 mm, and the recognition performance has good robustness to changes in wire size. However, when the wire diameter is reduced to 0.1 mm, the positioning results drift and become unstable due to insufficient point cloud deformation caused by the small contact area, making it difficult to achieve reliable contact position sensing. Therefore, the smallest stable recognition diameter that this method can currently achieve is 0.2 mm. This recognition accuracy covers the common guidewire sizes in vascular interventional surgery, verifying the feasibility of the proposed guidewire positioning method in microscale contact sensing scenarios.

[0133] When identifying the contact state between the guidewire and the visual-tactile sensor, the effectiveness of the real-time contact state identification method proposed in this paper is verified through a real-time status visualization interface.

[0134] As shown in Figure 4, (a)-(c) respectively show examples of recognition results under three contact states. The contact states output by the system are consistent with the actual operation process. In the figure, a status indicator is set on the right side of the interface to dynamically display the contact state between the guide wire and the visual-tactile sensor.

[0135] When the guidewire does not make contact with the visual tactile sensor, the indicator pointer points to the "not in contact" state;

[0136] When the guidewire comes into contact with the surface of the visual-tactile sensor but does not slide relative to it, the pointer on the indicator dial points to the "contact without rolling" state.

[0137] When the guidewire rolls along the surface of the visual-tactile sensor, the pointer on the indicator dial points to the "rolling" state.

[0138] As shown in Figure 5, as the applied external force increases, the bright area in the image gradually expands, the number of corresponding force-bearing markers increases, and the overall brightness increases synchronously, reflecting the continuous increase in the amplitude of the tangential displacement of the markers. Thus, it can be concluded that when k=63 and A=0.06, the system can obtain clear and stable visualization feedback in the grayscale image under the action of an external force of about 0.02N, and maintain good grayscale response consistency and resolution within the force range of 0.02 N–0.06 N.

[0139] In the real-time visualization feedback verification experiment simulating guidewire intervention, it was found that when the guidewire passes through the bending structure, its own bending causes the contact force between the guidewire and the visual-tactile sensor to remain at a high level for a long time. This results in the real-time visualization image based on tangential displacement being continuously highlighted, which may mask the superimposed local sudden displacement changes, thereby weakening the system's perception of abnormal contact force changes. In order to improve the sensitivity to local abnormal events while maintaining the continuity of overall force perception, contact force changes exceeding the normal range are displayed through abrupt change images. Therefore, a combination of real-time images and abrupt change images is used for visualization, as shown in Figure 6. (a)-(c) are the results when the guidewire does not encounter abnormal resistance after passing through the bending structure; (d)-(f) are the results when the guidewire encounters local resistance after passing through the bending structure.

[0140] The above description is merely a preferred embodiment of the present invention and is illustrative rather than restrictive. Those skilled in the art will understand that many changes, modifications, and even equivalents can be made within the spirit and scope defined by the claims of the present invention, all of which will fall within the protection scope of the present invention.

Claims

1. A method for sensing the contact state of a guidewire based on a visual-tactile sensor, characterized in that, The guidewire contact state sensing method includes the following steps: S1, extracting deformed point clouds from the complete tactile point cloud caused by guidewire contact or affected by local elastic conduction, and removing noise points; S2, by extracting the main axis of the denoised deformed point cloud, defining the marker point closest to the main axis as the sensing contact point, thereby achieving rapid localization of the guidewire contact area; S3, extracting the sensing contact point cloud, analyzing the changing characteristics of the sensing contact point cloud in adjacent frames, and identifying the contact state between the guidewire and the visual tactile sensor; S4, using a dual-channel visualization strategy, when the guidewire and the visual tactile sensor maintain reliable contact, detecting the small tangential displacement of the visual tactile sensor marker array in the x–y plane caused by the force on the guidewire, and constructing a real-time force visualization image and a sudden change response visualization image of the guidewire using grayscale mapping, respectively, to achieve real-time and accurate feedback on the force trend of the guidewire.

2. The method for sensing the contact state of a guidewire based on a visual-tactile sensor as described in claim 1, characterized in that, Step S1 specifically includes the following steps: S11, screening to obtain deformable point clouds: When the guide wire comes into contact with the visual-touch sensor, a local depression will be formed on the elastic surface of the visual-touch sensor. This deformation is mainly manifested as displacement along the z-axis in the point cloud. The z-axis displacement is used as a preliminary deformation criterion to screen out deformable point clouds; S12, filtering out noise points: The deformable point cloud is filtered using an isolated point filtering method based on local neighborhood statistics.

3. The method for sensing the contact state of a guidewire based on a visual-tactile sensor as described in claim 1, characterized in that, In step S11, during the filtering process, the midpoint of the reference frame is set. The coordinates in the z-axis direction are The point corresponding to the current frame The coordinates in the z-axis direction are Then its z-axis displacement can be expressed as: When the z-axis displacement of a point satisfies If the point is found to be deformed, it indicates that the point has undergone deformation and is identified as a deformation point caused by the contact of the guide wire. The point set obtained by this screening constitutes a preliminary deformed point cloud.

4. The method for sensing the contact state of a guidewire based on a visual-tactile sensor as described in claim 2, characterized in that, In step S12, during the removal process, a structure is constructed based on each point in the deformed point cloud. A local neighborhood window centered at the center is defined, with each local neighborhood window having a side length of 2r+1, where r is the neighborhood radius. All local neighborhood windows containing values ​​whose z-axis displacement is greater than a threshold are counted. The neighboring points are the valid neighboring points; calculate the points. The number of valid neighbors within a local neighborhood is used to determine the validity of a point. When the number of valid neighbor points exceeds half of the total number of points in the neighborhood, the point is considered valid. The region in question exhibits continuous deformation characteristics, and the points... The points are retained in the deformed point cloud; conversely, if the number of effective neighboring points is less than half, the points are removed. Isolated noise points are identified and removed to obtain the denoised deformed point cloud.

5. The method for sensing the contact state of a guidewire based on a visual-tactile sensor as described in claim 1, characterized in that, Step S2 specifically includes the following steps: S21, the physical contact between the guidewire and the visual-tactile sensor can be approximated as a straight line; based on the PCA method, the deformed point cloud is analyzed, and the eigenvector corresponding to the largest eigenvalue is selected as the principal axis of the deformed point cloud; S22, the perpendicular distance from each point in the deformed point cloud to the principal axis is calculated. To further determine the contact position of the guide wire on the surface of the visual-tactile sensor, the calculation formula is as follows: ;in, The center point of the deformed point cloud is the average coordinate of all points. The eigenvector corresponding to the largest eigenvalue; when a point in the deformable point cloud satisfies When the point is defined as the guide wire contact point perceived by the visual-tactile sensor, all points that meet the conditions constitute the sensing contact point cloud; S23, the selected sensing contact point cloud is directly assigned a specific color to distinguish the sensing contact point cloud from other point clouds, so that the contact position of the guide wire on the surface of the visual-tactile sensor can be intuitively presented in the point cloud, thereby realizing the positioning of the guide wire contact position.

6. The method for sensing the contact state of a guidewire based on a visual-tactile sensor as described in claim 1, characterized in that, The specific details of step S3 The process includes the following steps: S31, using the overlap rate R of the contact area between adjacent frames as a criterion to determine the state between the guidewire and the visual-tactile sensor, specifically: ;in, and These are the sets of perceptual contact point cloud indices for the current frame and the previous frame, respectively. The cardinality is the intersection of the point cloud indices of the contact regions in two adjacent frames; the contact state is determined according to the following rules: ;in, The minimum threshold number of sensing contact points. The contact consistency threshold is set when the number of perceived contact point clouds is less than the threshold. When the current point cloud size is insufficient to stably characterize the effective contact behavior between the guide wire and the visual-tactile sensor, the system directly determines it as a non-contact state; when the number of sensed contact point clouds is greater than or equal to the threshold... When, if the overlap rate R < If the guidewire and the visual-touch sensor are in a rolling state, it is determined that they are in a contact state without rolling. Otherwise, it is determined that they are in a contact state without rolling. S32, in order to avoid the process of the guidewire and the visual-touch sensor just making contact and gradually entering a stable contact state being mistakenly judged as a rolling state, a 3-frame tolerance mechanism is adopted: within the 3-frame stage, regardless of whether the contact area changes, the system determines that they are in a contact state without rolling.

7. The method for sensing the contact state of a guidewire based on a visual-tactile sensor as described in claim 1, characterized in that, Step S4 specifically includes the following steps: S41, when the guide wire makes reliable contact with the visual-tactile sensor, the guide wire is subjected to external force, which causes a small tangential displacement of the marker point array on the surface of the visual-tactile sensor in the x–y plane. The tangential displacement change of the marker points is detected to obtain the relative displacement of each marker point in the tangential direction; S42, a real-time force visualization image is constructed: the tangential displacement of each marker point in the marker point array is mapped to a grayscale value using a logarithmic mapping function, realizing an intuitive expression of the change of physical quantity to visual information. The logarithmic mapping function is: ;in, Let represent the tangential displacement amplitude of the marker point in the i-th row and j-th column of the marker point array. Tangential displacement The corresponding visualization grayscale value, k is the enhancement coefficient, and A is the upper limit reference value of the displacement mapping; S43, construct a visualization image of the mutation response: establish a dynamically updated reference displacement baseline for each marker point. ， The update method is an exponential moving average, and its update formula is: ;in, Forgetting factor, The initial value is determined by the displacement value when the visual-tactile sensor first detects a non-zero tangential displacement at the marker point; [The text abruptly ends here, likely due to an incomplete sentence or missing information.] Then, calculate the tangential displacement of the current frame marker point relative to... The difference ： ;when A large increase at this location can be considered an abnormal surge, potentially indicating localized obstruction anterior to the guidewire, increased friction, or a risk of contact with abnormal tissue; The same logarithmic mapping function as in step S42 is used to map the values ​​to grayscale to generate a mutation image.

8. The method for sensing the contact state of a guidewire based on a visual-tactile sensor as described in claim 7, characterized in that, Step S41 specifically includes the following steps: S411, constructing a reference point cloud using multi-frame statistics for marker point displacement calculation: With the system in a stable state without external forces, select several frames of marker point cloud data, statistically analyze the spatial coordinates of each marker point in the selected frames, and take the median as the reference coordinate of that marker point, thereby constructing a robust reference point cloud; S412, after constructing the reference point cloud, calculate the coordinate differences between the current frame point cloud and the reference point cloud to obtain the relative displacement of each marker point in the x, y, and z directions, as shown in the following formula: ;in, The coordinates of the marker point in the i-th row and j-th column of the current frame. For the coordinates of the marker point in the i-th row and j-th column of the reference point cloud, Let be the displacement of the marker point in row i and column j relative to the reference point cloud on the x, y, and z axes. If there is no point cloud data for the marker point in row i and column j in the current frame, then is recorded as . (0,0,0); after obtaining the relative displacements of each marker point in the x, y, and z directions, the tangential displacement of the marker point in the x–y plane is then determined. for: 。

Images