Catheter instrument identification method and system

Abstract

The application belongs to the technical field of interventional image recognition, and relates to a catheter instrument recognition method and system. Image data sequences of a catheter instrument in an interventional image are acquired. Multi-scale edge detection is performed on the image data, and a catheter instrument image region is divided into at least two categories according to pixel width differences. For each category, a bending curvature is extracted and a time sequence of the bending curvature is generated. A time sequence of a bending shape change index is calculated according to the time sequence of the bending curvature. The time sequence of the bending shape change index is input into an operation type classifier, and a probability distribution of each operation type is output. The probability distribution output by the operation type classifier at consecutive multiple time steps is input into an abnormality judgment classifier, and abnormal stress is judged and a warning is given when a normal operation state probability is lower than a preset threshold.

Description

Technical Field

[0001] This invention belongs to the field of interventional image recognition technology, specifically a catheter device recognition method and system. Background Technology

[0002] Interventional catheterization is a minimally invasive surgical procedure guided by imaging equipment, inserting a catheter through a blood vessel into a target site within the body for diagnosis or treatment. During the procedure, the surgeon controls the catheter's movement within the blood vessel by maneuvering such as pushing, retracting, and rotating. Existing technologies include methods for identifying and tracking catheters from interventional images. These methods utilize deep learning models to perform target detection and pixel-level segmentation of the catheter in X-ray images, identifying its position and outline within the image. Other methods analyze the catheter's morphological characteristics by extracting its centerline and calculating its curvature. Based on this analysis, thresholds are set for the catheter's morphological parameters; when the curvature exceeds the threshold, the catheter is deemed to be in a dangerous state.

[0003] However, the existing technologies described above still have room for improvement in detecting abnormal forces. When a catheter travels through a blood vessel, it undergoes significant bending deformation as it passes through tortuous segments. While this bending is substantial, it is a normal compliant deformation and does not pose a danger. The direct threshold determination method used in existing technologies cannot distinguish between the deformation of the catheter during normal passage through tortuous vessels and the deformation caused by abnormal resistance, easily leading to false alarms and failing to accurately reflect the true stress state of the catheter. Therefore, those skilled in the art are continuously focusing on improving the accuracy of detecting abnormal forces on interventional catheters. Summary of the Invention

[0004] To address the problems mentioned in the background section, this invention provides a catheter device identification method and system to solve the problem of low accuracy in detecting abnormal force on catheters.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a method for identifying catheter devices, comprising: Acquire image data sequences of catheter instruments in interventional imaging, wherein the image data sequences comprise multiple consecutive frames of interventional images; Multi-scale edge detection is performed on the image data to extract the edge response features of the catheter device at different scales. Based on the pixel width difference of the edge response features, the image region of the catheter device is divided into at least two categories. For each category, its morphological changes are tracked in the continuous multi-frame interventional images, the curvature of the category in each frame is extracted, and a time series of curvature is generated. Calculate the time series of the bending morphology change index based on the time series of the bending curvature; The time series of the bending morphology change index is input into a pre-trained operation type classifier, which outputs the probability distribution of the catheter instrument for each operation type. The probability distribution output by the operation type classifier at multiple consecutive time steps is input into a pre-trained anomaly detection classifier. When the probability of the anomaly detection classifier outputting the normal operation state is lower than a preset threshold, the catheter device is determined to be in an abnormal stress state and an early warning message is output. When the catheter device is under abnormal stress, its morphological changes deviate from the kinematic laws of normal operation, and the probability distribution output by the operation type classifier at multiple consecutive time steps exhibits a category confidence decay characteristic.

[0006] Optionally, the image region of the catheter device is divided into at least two categories based on the pixel width difference of the edge response features, specifically including: The image data is subjected to multi-scale image transformation, and edge detection is performed at each scale to obtain a multi-scale edge response map; At the same spatial location, the pixel width values ​​of the edge response at different scales are statistically analyzed, and the corresponding catheter instrument image regions at spatial locations where the pixel width values ​​change along the scale direction and meet the preset jump conditions are divided into different categories.

[0007] Optionally, the at least two categories include a first category and a second category, wherein the edge response pixel width of the first category is smaller than the edge response pixel width of the second category.

[0008] Optionally, a time series of the curvature is generated, specifically including: For each category, skeleton extraction is performed in each frame of the interventional image to obtain the centerline path of that category; Multiple path points are sampled at equal intervals along the centerline path, and the curvature at each path point is calculated. The curvature at each of the path points is arranged in order of distance from the centerline path to form the curvature distribution of this category in a single frame image; Arrange the curvature distribution of multiple consecutive frames in time sequence to generate a time series of the bending curvature corresponding to that category.

[0009] Optionally, before inputting the time series of the bending morphology change index into a pre-trained operation type classifier, the method further includes constructing a training sample set for the operation type classifier: Obtain historical interventional image data sequences and their corresponding operation type labels, wherein the operation type labels include push operation, pullback operation, rotation operation and stationary operation; Perform the multi-scale edge detection and category classification on the historical interventional image data sequence to obtain the curvature time series of each category; The time series of the bending morphology change index is calculated based on the bending curvature time series and used as the feature vector of the training samples. The operation type classifier is trained using the corresponding operation type label as the training label.

[0010] Optionally, the operation type classifier is a temporal neural network classifier, which is used for: Receive the time-series data of the bending morphology change index; The time series data is subjected to time series feature extraction, and the time series features include the statistics of the bending morphology change index at each time step and the change trend between the bending morphology change index of adjacent time steps. The basic attention weights are assigned based on the distance of each path point from the far end of the category. The closer the path point is to the far end, the higher the basic attention weight. The basic attention weights are then dynamically adjusted based on the curvature change at each path point. The far end is the end of the path that penetrates into the body of the category. The probability distribution of the catheter device for each operation type is output based on the extracted temporal features and the adjusted attention weights.

[0011] Optionally, determining that the catheter device is under abnormal stress and outputting a warning message specifically includes: The anomaly detection classifier takes the probability distribution sequence output by the operation type classifier at multiple consecutive time steps and the category entropy value of the probability distribution sequence as input, the category entropy value is the Shannon entropy value of the probability distribution at each time step, and the anomaly detection classifier outputs a binary classification of normal operation state and abnormal force state. When the probability of the anomaly classifier outputting a normal operating state is lower than the preset threshold and the category entropy value is higher than the preset entropy value threshold, the catheter device is determined to be in an abnormal stress state. The preset threshold is dynamically adjusted based on the confidence level of the current operation type output by the operation type classifier. The higher the confidence level of the current operation type, the higher the value of the preset threshold.

[0012] Optionally, the training sample set for constructing the anomaly detection classifier includes: Historical interventional image data sequences and their corresponding stress state labels are obtained, the stress state labels include normal operation state and abnormal stress state; the probability distribution sequence and category entropy value output by the operation type classifier at multiple consecutive time steps are used as feature inputs of training samples, and the corresponding stress state labels are used as training labels to train the anomaly determination classifier.

[0013] Optionally, the anomaly classification classifier also takes the time-series cross-correlation value between the time series of the bending morphology change index of different categories as input; When the time-series cross-correlation value is lower than a preset correlation threshold, the preset threshold shifts in the direction of increasing value.

[0014] A catheter device identification system, comprising: The image acquisition module is used to acquire image data sequences of catheter instruments in interventional images, wherein the image data sequences contain multiple consecutive frames of interventional images; The category segmentation module is used to perform multi-scale edge detection on the image data, extract the edge response features of the catheter device at different scales, and divide the image region of the catheter device into at least two categories based on the pixel width difference of the edge response features. The time-series tracking module is used to track the morphological changes of each category in the continuous multi-frame interventional images, extract the curvature of the category in each frame, and generate a time series of curvature. An index calculation module is used to calculate a time series of the bending morphology change index based on the time series of the bending curvature. The operation recognition module has a built-in pre-trained operation type classifier, which is used to input the time series of the bending morphology change index into the operation type classifier and output the probability distribution of the catheter instrument in each operation type. An anomaly detection module is used to input the probability distribution output by the operation type classifier at multiple consecutive time steps into a pre-trained anomaly detection classifier. When the probability of the anomaly detection classifier outputting the normal operation state is lower than a preset threshold, the catheter device is determined to be in an abnormal stress state and an early warning message is output. In the anomaly determination module, when the catheter device is under abnormal stress, its morphological changes deviate from the kinematic laws of normal operation, and the probability distribution output by the operation type classifier at multiple consecutive time steps exhibits a category confidence decay characteristic.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention quantifies the dynamic morphological changes of catheters during continuous interventional procedures by extracting the curvature of catheter instruments from interventional images and calculating the inter-frame curvature changes. Based on this, the invention identifies the current operation type using an operation type classifier and further utilizes the probability distribution output by this classifier at continuous time steps for anomaly detection. During normal operation, catheter morphological changes follow defined kinematic laws, and the classifier can stably output high-confidence identification results. However, when the catheter is under abnormal stress, its morphological changes deviate from normal kinematic laws, and the probability distribution output by the classifier exhibits a category confidence decay characteristic. Compared to existing technologies that directly set thresholds for morphological parameters for judgment, this invention utilizes the temporal variation characteristics of the classifier's output probability distribution to identify abnormal stress, effectively distinguishing between normal catheter conformal deformation and abnormal stress deformation, reducing false alarm rates, and improving the accuracy and reliability of abnormal catheter stress detection during interventional procedures. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the overall method flow of the present invention; Figure 2 This is a schematic diagram of the overall system flow of the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] like Figures 1 to 2 As shown, the present invention provides a catheter device identification method, comprising: Acquire image data sequences of catheter instruments in interventional imaging, wherein the image data sequences comprise multiple consecutive frames of interventional images; Multi-scale edge detection is performed on the image data to extract the edge response features of the catheter device at different scales. Based on the pixel width difference of the edge response features, the image region of the catheter device is divided into at least two categories. For each category, its morphological changes are tracked in the continuous multi-frame interventional images, the curvature of the category in each frame is extracted, and a time series of curvature is generated. Calculate the time series of the bending morphology change index based on the time series of the bending curvature; The time series of the bending morphology change index is input into a pre-trained operation type classifier, which outputs the probability distribution of the catheter instrument for each operation type. The probability distribution output by the operation type classifier at multiple consecutive time steps is input into a pre-trained anomaly detection classifier. When the probability of the anomaly detection classifier outputting the normal operation state is lower than a preset threshold, the catheter device is determined to be in an abnormal stress state and an early warning message is output. When the catheter device is under abnormal stress, its morphological changes deviate from the kinematic laws of normal operation, and the probability distribution output by the operation type classifier at multiple consecutive time steps exhibits a category confidence decay characteristic.

[0019] Specifically, one embodiment of the present invention provides a catheter device identification method, which dynamically analyzes the morphological changes of catheter devices based on interventional imaging sequences, thereby identifying the operation type and determining abnormal stress state.

[0020] This invention acquires image data sequences of catheter devices from interventional imaging, comprising multiple consecutive frames of interventional images. Interventional images are real-time fluoroscopic images acquired during the interventional procedure, in which the catheter device appears as a thin, elongated linear structure. The acquired image data sequence serves as the data foundation for all subsequent analysis steps; its continuous multi-frame characteristic allows this invention to capture the morphological changes of the catheter device over time.

[0021] Multi-scale edge detection is performed on each frame of the image data sequence to extract the edge response features of the catheter instruments at different scales. Multi-scale edge detection involves performing multi-scale image transformations on the image to extract the edge information of the catheter instruments at each scale. Since different segments of the catheter instruments exhibit different pixel widths in the image—for example, thin segments appear as narrow lines and thick segments as wide lines—the pixel width of the edge response at the same spatial location differs at different scales. Based on this difference in pixel width, the image region of the catheter instruments is divided into at least two categories, ensuring that segments with different pixel widths are grouped into different categories. This categorization allows subsequent morphological tracking to be performed separately for segments with different diameters, avoiding the superposition or confusion of morphological features from different segments.

[0022] For each identified category, morphological changes within that category are tracked across multiple consecutive frames of interventional images. Specifically, a centerline path for that category is extracted from each frame. This centerline path is a single-pixel-width line along the length of the catheter, representing the spatial orientation of the catheter in that frame. Multiple path points are sampled at equal intervals along this centerline path, and the curvature at each path point is calculated. The curvature reflects the degree of curvature of the centerline path at that point. The curvatures at all path points in each frame are arranged sequentially according to the path, forming the curvature distribution for that frame. The curvature distributions across multiple consecutive frames are then arranged temporally to form a curvature time series for that category. This curvature time series simultaneously includes the spatial location information of each path point, the temporal information of each frame, and the curvature values ​​of each point at each time step, comprehensively recording the dynamic evolution of the category's morphology in both spatial and temporal dimensions.

[0023] Based on the obtained time series of curvature for each category, a time series of the curvature morphology change index is calculated. For each frame of interventional image, the curvature of each sampling point in the current frame is compared with the curvature of the corresponding sampling point in the previous frame to obtain the amount of curvature change at each sampling point. The absolute values ​​of the curvature changes at all sampling points are accumulated along the centerline path to obtain the curvature morphology change index for the current frame. This index characterizes the magnitude of change in the overall morphology of that category from the previous frame to the current frame. Because absolute values ​​are accumulated, curvature changes in opposite directions occurring in different segments of the catheter will not cancel each other out, thus truly reflecting the severity of the overall morphological changes of the catheter. The same processing is performed on all frames, and the curvature morphology change indices of each frame are arranged in chronological order to form a time series of the curvature morphology change index. Thus, the morphological changes of catheter instruments in the original images are quantified into a numerical sequence that changes over time, which serves as the input feature for subsequent operation type identification.

[0024] The time series of the bending morphology change index is input into the operation type classifier. This operation type classifier is a pre-trained classification model; its input is the time series of the bending morphology change index, and its output is the probability distribution of the catheter instrument for each operation type. Since different operation types—push, pull, and rotate—correspond to different morphology change patterns, the operation type classifier, after training, can determine the current operation type based on the time series characteristics of the input bending morphology change index and output the judgment result in the form of a probability distribution. The probability value corresponding to each type in the probability distribution reflects the classifier's judgment of the current operation type and its confidence level.

[0025] The probability distribution output by the operation type classifier over multiple consecutive time steps is input into the anomaly detection classifier. This anomaly detection classifier is a pre-trained binary classification model. Its input is a sequence of probability distributions over multiple consecutive time steps, and its output is the probability value corresponding to whether the current state belongs to the normal operation state or the abnormal stress state. When the probability of the anomaly detection classifier outputting the normal operation state is lower than a preset threshold, the catheter device is determined to be in an abnormal stress state, and a warning message is output.

[0026] In the above method, the following correlation exists between the output confidence change of the operation type classifier and the catheter stress state: When the catheter is under normal stress, its morphology changes according to the doctor's operational intention, and its morphological change pattern is consistent with the normal operation pattern in the operation type classifier training data. Therefore, the operation type classifier can stably output a high-confidence probability distribution in continuous time steps. When the catheter is under abnormal stress, such as when the tip experiences abnormal resistance from the blood vessel wall or the catheter becomes stuck in the blood vessel, the morphological movement of the catheter will be unexpectedly disturbed, and its morphological change deviates from the kinematic law of normal operation. At this time, the operation type classifier cannot stably identify a specific operation type in continuous time steps, and its output probability distribution tends to be dispersed among different operation types, the difference in probability values ​​between different types narrows, and the classifier's recognition confidence is significantly reduced. The anomaly detection classifier is based on this principle, and determines whether the catheter is under abnormal stress by analyzing the temporal changes in the output probability distribution of the operation type classifier. Compared to existing technologies that directly set thresholds for morphological parameters such as the degree of bending to determine anomalies, this invention indirectly determines anomalies by observing the stability of the classifier's recognition results. This effectively distinguishes between the deformation of a catheter when it passes through a tortuous blood vessel normally and the deformation when it is subjected to abnormal force, thus avoiding false alarms triggered by the bending itself caused by the anatomical structure of the blood vessel.

[0027] The image region of the catheter device is divided into at least two categories based on the pixel width difference of the edge response features, specifically including: The image data is subjected to multi-scale image transformation, and edge detection is performed at each scale to obtain a multi-scale edge response map; At the same spatial location, the pixel width values ​​of the edge response at different scales are statistically analyzed, and the corresponding catheter instrument image regions at spatial locations where the pixel width values ​​change along the scale direction and meet the preset jump conditions are divided into different categories.

[0028] Specifically, in one embodiment of the present invention, the multi-scale edge detection and category classification steps are described in detail below.

[0029] After acquiring the image data sequence of catheter instruments in the interventional imaging, multi-scale image transformation is performed on each frame of the interventional image in the image data sequence. Multi-scale image transformation refers to transforming the original image according to different scale factors to obtain representations of the original image at different resolutions. In one embodiment of the present invention, multi-scale image transformation is implemented using a Gaussian pyramid approach. The Gaussian pyramid constructs a set of image layers with progressively decreasing resolution by performing Gaussian filtering and downsampling on the original image. Specifically, the original image is the 0th layer of the Gaussian pyramid. Gaussian filtering and downsampling of the 0th layer image yields the 1st layer image; Gaussian filtering and downsampling of the 1st layer image yields the 2nd layer image, and so on, forming a multi-scale image layer. Gaussian filtering uses a Gaussian kernel convolved with the image, and the standard deviation of the Gaussian kernel is... , The value of determines the smoothness of the filter. The larger the value of , the stronger the smoothing effect of the Gaussian filter and the stronger the noise suppression capability, but at the same time it will blur the edge details of the catheter instrument. The smaller the value of , the more complete the edge details are preserved, but the higher the sensitivity to noise. In one embodiment of the present invention, The value ranges from 0.5 to 1.5, for example... If the noise level of the interventional image is high, the volume can be increased accordingly. The value of is adjusted to enhance the smoothing effect; if the edges of the catheter instrument are blurry in the image, the value can be reduced accordingly. The value is chosen to preserve more edge details. The specific value can be adjusted by those skilled in the art based on the actual signal-to-noise ratio of the interventional images and the imaging clarity of the catheter instruments. Standard deviation of the Gaussian kernel. This determines the width of the Gaussian kernel. When At that time, the effective radius of the Gaussian kernel covering approximately 95% of the energy is approximately The value of 1 pixel can effectively suppress quantum noise and electronic noise commonly found in interventional images, while preserving the edge features of catheter instruments in the image. It is a commonly used default parameter in the field of image processing. The value can be scaled according to the image resolution; it increases accordingly when the image resolution is high. To cover sufficient physical space.

[0030] In this embodiment, a Gaussian pyramid is constructed with four layers, and the scale factors of each layer are 1, 2, 4, and 8, respectively. The scale factor is the ratio of the pixel size of the current layer image to the pixel size of the original image. The edge response characteristics of the catheter instruments in each layer image exhibit different characteristics at different scales. The edge response of the thin-diameter structure is clear at smaller scales, while the edge response of the thick-diameter structure is more prominent at larger scales.

[0031] Edge detection is performed on images at various scales to obtain edge response maps for each scale. The purpose of edge detection is to identify locations in the image where grayscale values ​​change drastically, corresponding to the boundaries of catheter instruments. In this embodiment, the Canny edge detection algorithm is used. Canny edge detection first smooths the image using a Gaussian filter to reduce noise interference, and then calculates the gradient magnitude and gradient direction for each pixel in the image. The gradient magnitude reflects the drasticness of the grayscale change at that pixel, and the gradient direction reflects the direction of the grayscale change. Non-local maxima suppression is applied to the gradient magnitude, retaining the pixels with the largest local gradient magnitude and suppressing non-local maxima points. Then, a double thresholding method is used to connect the edge points to obtain a binarized edge response map. In the edge response map, a value of 1 for an edge pixel indicates that the point belongs to the edge, and a value of 0 for a non-edge pixel indicates that the point does not belong to the edge. The above edge detection process is performed at each scale to obtain the edge response map corresponding to each scale.

[0032] The multi-scale edge response map constitutes a three-dimensional data structure, with its three dimensions being the row coordinates, column coordinates, and scale dimension of the image plane. For the same spatial location on the image plane, the pixel width value of the edge response in the edge response map at different scales is statistically analyzed. The pixel width value refers to the number of pixels belonging to the edge response region along a cross-section perpendicular to the extension direction of the catheter at that spatial location. Specifically, for the edge response map at each scale, the edge response region is scanned along the cross-sectional direction at the target spatial location, and the number of consecutive edge response pixels in that direction is recorded; this number is the pixel width value at that scale. Because the catheter exhibits different edge response widths at different scales, the pixel width value at the same spatial location varies across different scales.

[0033] The image regions of catheter devices corresponding to spatial locations where the pixel width value meets a preset jump condition when changing along the scale direction are classified into different categories. The preset jump condition is used to determine whether the pixel width value at a certain spatial location has changed significantly. In this embodiment, taking a certain spatial location of a catheter device as an example, its pixel width values ​​at scale factors of 1, 2, 4, and 8 are measured to be 3 pixels, 4 pixels, 4 pixels, and 5 pixels, respectively. The pixel width value remains relatively stable across the scales, indicating that this location belongs to a catheter segment of the same thickness level, and this location is classified into the same category. For another spatial location, its pixel width values ​​at scale factors of 1, 2, 4, and 8 are measured to be 3 pixels, 4 pixels, 8 pixels, and 10 pixels, respectively. The pixel width value undergoes a significant jump as the scale factor changes from 2 to 4. This jump indicates that this location exhibits thin-diameter edge features at smaller scales and thick-diameter edge features at larger scales. The edge responses at different scales reveal that the catheter device at this location has multiple thickness levels of structural superposition. If a location is determined to meet a preset transition condition, the corresponding catheter device image region is classified into different categories. The specific method for determining the preset transition condition is as follows: calculate the difference in pixel width values ​​between adjacent scales; if the difference exceeds a preset transition threshold, the transition condition is considered met. The preset transition threshold can be set according to the resolution of the interventional image and the actual size range of the catheter device. In this embodiment, the preset transition threshold is 3 pixels. Alternatively, the preset transition condition can also be that the ratio of pixel width values ​​between adjacent scales exceeds a preset ratio threshold. For example, if the pixel width value at a certain scale is more than twice the pixel width value at the adjacent smaller scale, the transition condition is considered met.

[0034] Through the above processing, the image region of the catheter device is divided into at least two categories. Each category corresponds to a pixel width level, and different categories represent segments of different diameters within the catheter device. For example, when both guidewires and catheters are present in the interventional image, the guidewire, with its narrower pixel width, is classified into the first category, while the catheter, with its wider pixel width, is classified into the second category. The two categories occupy different spatial regions in the image. Subsequent morphological tracking and analysis steps will be performed independently for each category, thus avoiding feature distortion caused by analyzing the morphological features of devices with different diameters together.

[0035] In this embodiment, the edge response pixel width of the first category is smaller than that of the second category. The first category corresponds to the thinner segments of the catheter instrument, such as the soft tip of the guidewire or catheter, while the second category corresponds to the thicker segments of the catheter instrument, such as the main body of the catheter. This classification allows subsequent morphological analysis to employ differentiated analysis strategies for segments of different diameters.

[0036] The at least two categories include a first category and a second category, wherein the edge response pixel width of the first category is smaller than the edge response pixel width of the second category.

[0037] Specifically, in one embodiment of the present invention, regarding the classification results of the first category and the second category, the at least two categories identified above include the first category and the second category. The edge response pixel width of the first category is smaller than that of the second category. The first category and the second category are relative classifications based on the pixel width relationship, and the specific number of classifications is determined according to the actual composition of the catheter device in the interventional image. Taking a typical interventional surgery scenario as an example, when both guidewire and catheter are present in the interventional image, the guidewire appears with a narrower pixel width in the X-ray fluoroscopy image and is classified as the first category, while the catheter appears with a wider pixel width in the X-ray fluoroscopy image and is classified as the second category. When only a single catheter device is present in the interventional image, the diameter of the tip of the catheter device gradually decreases, appearing with a narrower pixel width in the image and being classified as the first category, while the diameter of the main body of the catheter device is larger, appearing with a wider pixel width in the image and being classified as the second category. Through the above classification, each category contains catheter device regions in the image with similar pixel width levels.

[0038] Generating a time series of curvature, specifically including: For each category, skeleton extraction is performed in each frame of the interventional image to obtain the centerline path of that category; Multiple path points are sampled at equal intervals along the centerline path, and the curvature at each path point is calculated. The curvature at each of the path points is arranged in order of distance from the centerline path to form the curvature distribution of this category in a single frame image; Arrange the curvature distribution of multiple consecutive frames in time sequence to generate a time series of the bending curvature corresponding to that category.

[0039] Specifically, in one embodiment of the present invention, regarding the generation of the curvature time series, after the above-mentioned category division is completed, the following processing is performed for each category.

[0040] For a given category, in each frame of the interventional image, a skeleton is extracted from the image region of that category to obtain the centerline path for that category. Skeleton extraction refers to the process of peeling away edge pixels layer by layer from the binarized image region until only a single-pixel-width connected line remains. In one embodiment of the present invention, skeleton extraction is implemented using a morphological thinning algorithm. The basic logic of the morphological thinning algorithm is as follows: for each foreground pixel in the image region, the distribution pattern of its eight neighboring pixels is checked. If deleting this pixel will not disrupt the region's connectivity or shorten the region's endpoints, the pixel is deleted from the foreground set. The above checking process is repeated until all foreground pixels no longer meet the deletion conditions, at which point the iteration terminates, and the remaining foreground pixels constitute the skeleton. The pixels on the skeleton are connected sequentially to form the centerline path, and the extension direction of the centerline path is consistent with the extension direction of the catheter instrument in the image. The centerline path is represented in the form of an ordered set of points: , , , ,in Indicates the first [number] on the centerline path The coordinates of each pixel in the image coordinate system, with subscripts. Arranged in chronological order of the paths. This represents the total number of pixels along the centerline path. The centerline path starts on the side of the catheter instrument near the puncture site and ends on the side of the catheter instrument inserted deep into the body.

[0041] After obtaining the ordered set of points along the centerline path, multiple path points are sampled at equal intervals along the centerline path. Since the original pixels along the centerline path are discretely distributed on the image grid, the Euclidean distance between adjacent pixels is not a constant value. Directly using the original pixels as path points would result in uneven spatial distribution of the sampled points. Therefore, equal-arc-length resampling along the centerline path is necessary. Specifically, the total arc length of the centerline path is calculated. The arc length is obtained by summing the Euclidean distances between adjacent pixels: ; in For pixels and The Euclidean distance between them is calculated using the following formula: ; in , and , Each pixel and The horizontal and vertical coordinates in the image coordinate system.

[0042] To obtain the total arc length Then determine the number of samples. The arc length interval for equidistant sampling is: ; From the starting point Begin by sequentially obtaining distances from the starting point along the centerline path. , , , , The path point location. For the sampling point location located between two original pixels, its coordinates are determined using linear interpolation. Assume the sampling point is located at an original pixel. and Between, sampling point distance The arc length is , and The Euclidean distance between them is: ; The interpolation weights are: ; The x and y coordinates of the sampling points are as follows: ; ; in , and , Each pixel and The horizontal and vertical coordinate values ​​are obtained. Through the above equal arc length resampling, uniformly distributed along the centerline path are obtained. Path points: , , , ,in Located near the starting point, Located near the endpoint, the arc length between adjacent path points is 1. .

[0043] For each sampling path point Calculate the curvature at that point. The curvature characterizes the degree of bending of the centerline path at a given point, and its value is equal to the rate of change of the tangent direction angle along the arc length at that point. Under discrete data conditions, the curvature is calculated as follows: For sampling path points... Take its previous path point and the next waypoint Construct vector and , Indicates the path is in The forward direction of the location Indicates the path is in The backward direction at the location. Let... The coordinates are , , The coordinates are , , The coordinates are , Then the vector The x and y coordinates are as follows: ; ; vector The x and y coordinates are as follows: ; ; calculate and The angle between : ; in The dot product of two vectors: ; and The Euclidean norms of the two vectors are: ; ; included angle The value range is 0 to This reflects the path in The change in direction at a point. Curvature. Through the included angle Divide by the corresponding arc length interval to obtain the arc length interval under the condition of equal distance sampling. ,therefore: ; For sampling points at the path endpoints and Since there are no forward or backward neighboring points, its curvature is obtained by extrapolating the curvature of the nearest points, i.e., we take: ; ; The curvature at each sampling path point is arranged in order of distance from the centerline path to form the curvature distribution of this category in a single frame image: , , , ; Subscript Indicates the frame number. Indicates the first The first frame of the image The curvature at each sampling path point It is Dimensional vector.

[0044] For each frame in a series of consecutive interventional images, the above-mentioned skeleton extraction, equidistant sampling, and curvature calculation processes are performed to obtain the curvature distribution corresponding to each frame. , , , ,in The total number of frames. The curvature distribution of multiple consecutive frames is arranged temporally to generate a time series of the curvature corresponding to that category: , , , ; The time series of the curvature Curvature distribution of each frame For one 3D row vectors The row vectors are arranged according to the frame number to form a OK A matrix of columns, the first column in the matrix Line number The element of the column is the first Frame number Bending curvature value at each sampling path point .

[0045] The skeleton extraction, equidistant sampling, and curvature calculation processes described above are performed independently for each of the identified categories. When there are two categories, a first category and a second category, the curvature time series of the first category is obtained separately. Second category of bending curvature time series .

[0046] Before inputting the time series of the bending morphology change index into the pre-trained operation type classifier, the method further includes constructing a training sample set for the operation type classifier: Obtain historical interventional image data sequences and their corresponding operation type labels, wherein the operation type labels include push operation, pullback operation, rotation operation and stationary operation; Perform the multi-scale edge detection and category classification on the historical interventional image data sequence to obtain the curvature time series of each category; The time series of the bending morphology change index is calculated based on the bending curvature time series and used as the feature vector of the training samples. The operation type classifier is trained using the corresponding operation type label as the training label.

[0047] Specifically, in one embodiment of the present invention, regarding the training of the operation type classifier, before inputting the time series of the bending morphology change index into the pre-trained operation type classifier, it is necessary to construct a training sample set to train the classifier.

[0048] Historical interventional imaging data sequences and their corresponding operation type annotations are obtained. These sequences are fluoroscopic image records collected from previous interventional surgical cases, with each sequence covering at least one complete procedure. Simultaneously, a specialist manually annotates each frame or consecutive frame sequence, labeling it with the specific operation type. Operation type annotations include four types: pushing, withdrawing, rotating, and stationary. Pushing refers to the physician advancing the catheter along the blood vessel direction, with the catheter tip moving towards the lesion; withdrawing refers to the physician pulling the catheter back outwards, with the catheter tip moving towards the puncture site; rotating refers to the physician rotating the catheter, causing it to twist around its axis; and stationary refers to the catheter remaining stationary without any active manipulation by the physician. Annotation is done by assigning a category label to each frame corresponding to the operation type, or by assigning the same category label to several consecutive frames.

[0049] Multi-scale edge detection and classification are performed on historical interventional image data sequences. The specific implementation method of multi-scale edge detection and classification is completely consistent with the processing method described above for current interventional images. Multi-scale image transformation is performed on each frame of the historical images, and edge detection is performed at each scale to obtain multi-scale edge response maps. The pixel width values ​​of the edge responses at different scales are counted at the same spatial location. The catheter instrument image regions corresponding to the spatial locations where the pixel width values ​​change along the scale direction and meet the preset jump conditions are classified into different categories to obtain the image regions of each category.

[0050] For each category identified in the historical imagery, a time series of curvature for each category is obtained. The curvature time series is obtained as follows: For each category, skeleton extraction is performed on each frame of the image to obtain the centerline path for that category. The centerline path is represented as an ordered set of points. , , , ,in Indicates the first [number] on the centerline path The coordinates of each pixel This represents the total number of pixels. Sampling is performed at equal intervals along the centerline path. There are path points, and the arc length interval for equidistant sampling is... ,in The total arc length of the centerline path: ; For sampling path points Take its previous path point and the next waypoint Construct vector and .set up The coordinates are , , The coordinates are , , The coordinates are , Then the vector and The x and y coordinates are respectively: ; ; ; ; Calculate the angle between two vectors: ; The curvature at each path point is The curvature at each path point is arranged in order of distance from the centerline path, forming the curvature distribution of this category in a single frame image: , , , ; Subscript Indicates the frame number. Indicates the first The first frame of the image The curvature at each sampling path point. For each frame in a series of consecutive images, the above processing is performed to obtain the curvature distribution corresponding to each frame. , , , ,in For the total number of frames, generate a time series of the curvature corresponding to this category: , , , ; The time series of curvature time series is used to calculate the curvature morphology change index. For each image frame, for each category, the curvature of each sampling point in the current frame is calculated. Curvature of the corresponding sampling point in the previous frame Perform interpolation to obtain the curvature change at each sampling point. Accumulate the absolute values ​​of the curvature changes at each sampling point along the centerline path to obtain the curvature change index of the current frame. ; For the first frame, since there is no previous frame, its bending shape change index is... Set it directly to 0. Arrange the bending shape change index of multiple consecutive frames in chronological order to generate a time series of the bending shape change index: , , , ; The time series of the obtained bending morphology change index is used as the feature vector of the training samples, and the corresponding operation type label is used as the training label to train the operation type classifier. Specifically, a sliding window method is used to extract training samples from the time series of the bending morphology change index. Let the length of the time window be... Starting from the beginning of the time series, take the first to the last position sequentially. Frames 2 to 3 Frames, 3rd to 4th Frames, and so on, form a feature vector for each training sample based on the exponential sequence of bending morphology changes within each time window. Each training sample corresponds to an operation type label, which is the operation type annotation corresponding to the center time of the time window or most frames within the time window. All training samples are input into the temporal neural network classifier to be trained, and the cross-entropy loss between the operation type probability distribution output by the classifier and the true labels is calculated. ; in One-hot encoding for the real label, The first output of the classifier The probability values ​​of each operation type are determined. Backpropagation is used to update network parameters, and the training is iterated multiple times until the loss function converges, resulting in a trained operation type classifier. When multiple categories exist, the time series of bending morphology change indices for each category are concatenated into a multi-channel feature vector as input to the training samples, or they are input into separate network branches for processing and then fused. The network parameters of each branch are updated synchronously during training.

[0051] The operation type classifier is a temporal neural network classifier, which is used for: Receive the time-series data of the bending morphology change index; The time series data is subjected to time series feature extraction, and the time series features include the statistics of the bending morphology change index at each time step and the change trend between the bending morphology change index of adjacent time steps. The basic attention weights are assigned based on the distance of each path point from the far end of the category. The closer the path point is to the far end, the higher the basic attention weight. The basic attention weights are then dynamically adjusted based on the curvature change at each path point. The far end is the end of the path that penetrates into the body of the category. The probability distribution of the catheter device for each operation type is output based on the extracted temporal features and the adjusted attention weights.

[0052] Specifically, in one embodiment of the present invention, regarding the network structure of the operation type classifier, the classifier is a temporal neural network classifier, and its processing flow is as follows.

[0053] The temporal neural network classifier receives input data, which includes the distribution of curvature values ​​along the path at each sampling path point in each frame, and a time series of the curvature morphology change index calculated based on this distribution. Specifically, for a certain category in each frame, the curvature distribution at each sampling path point is as follows: , , , This distribution records the data in that frame. The curvature values ​​corresponding to each sampling path point, among which Indicates the first The first frame of the image The curvature at each sampling path point. The curvature distribution of each frame constitutes a time series of curvature. , , , The time series of the bending morphology change index is as follows: , , , The bending shape change index of each frame Obtain it in the following ways: ; That is, the sum of the absolute values ​​of the curvature changes at each sampling path point.

[0054] In the input data, the curvature distribution of each frame preserves the spatial location information of each sampled path point, enabling the network to distinguish and process different path points separately in the spatial dimension. The time series of the curvature change index provides temporal information on the severity of the overall morphological changes of the catheter instrument, allowing the network to capture the dynamic evolution of morphological changes in the temporal dimension. Together, these two elements constitute a feature tensor containing spatial and temporal dimensions, where the spatial dimension corresponds to each sampled path point along the centerline path, and the temporal dimension corresponds to each consecutive frame of imagery.

[0055] This temporal neural network classifier extracts temporal features from the input time series data. These features include statistics on the bending morphology change index at each time step and the trend of change between bending morphology change indices at adjacent time steps. Let the bending morphology change index sequence within a time window be... , , , ,in Given the time window length, calculate the mean: ; variance: ; Maximum value , , , minimum value , , , The trend of change includes the difference sequence between adjacent time steps. , , , And its statistical characteristics, such as the mean and variance of the differences. Temporal feature extraction is achieved through recurrent or convolutional layers in temporal neural networks. Recurrent layers use long short-term memory network units or gated recurrent units to capture long-term dependencies in time series, while convolutional layers use one-dimensional convolutional kernels to extract local features in the time dimension.

[0056] This temporal neural network classifier assigns basic attention weights based on the distance of each path point from the far end of its category. After completing the aforementioned category division and path point sampling, each sampled path point is arranged sequentially along the centerline path from the starting point to the ending point, where the far end is the end of the path that extends into the body of that category, i.e., the endpoint. Location. For sampling path points. Its distance to the far end is ,in This is the total arc length of the centerline path for this category. The arc length interval for equidistant sampling. , , , . A smaller value indicates that the path point is closer to the far end. The base attention weights for each path point are determined based on... In the allocation process, pathpoints closer to the distant endpoint receive higher base attention weights. These base attention weights are calculated as follows: ; in This is a preset attenuation coefficient used to control the rate at which the weight decays with distance. When When positive values ​​are taken, path points closer to the far end receive higher basic attention weights, while path points closer to the starting point receive lower basic attention weights.

[0057] Based on the basic attention weights, this temporal neural network classifier further dynamically adjusts the basic attention weights according to the curvature change at each path point. For each frame of the intervention image, the curvature change at each sampling path point is: ; in For the current frame, the first The curvature at each sampling path point This represents the curvature at the corresponding sampling path point in the previous frame. The absolute value of the change in curvature. This reflects the magnitude of the curvature change at that path point. Calculate the sum of the absolute values ​​of the curvature changes at all sampled path points: ; For each path point, calculate the proportion of the absolute value of its curvature change to the total: ； This ratio reflects the relative contribution of the pathpoint to the morphological changes in the current frame. The dynamically adjusted final attention weight is: ; When the curvature change at a certain path point is large, its When the value is large, the final attention weight of that path point is increased relative to the base weight; when the curvature change of a path point is small, its final attention weight is decreased relative to the base weight. The dynamically adjusted attention weights allow the classifier to focus on the local regions with the most dramatic morphological changes in the current frame while paying attention to distant regions.

[0058] This temporal neural network classifier outputs the probability distribution of catheter instruments for each operation type based on the extracted temporal features and adjusted attention weights. Specifically, the feature vector output by the temporal feature extraction layer is weighted and fused with the attention weights of each path point to obtain a weighted global feature representation. Let the feature vector output by the temporal feature extraction layer be... The attention weight vector is , , , The local features corresponding to each path point are: The global features after weighted fusion are: ; The weighted fused global features The input is a classification output layer, which consists of a fully connected layer and a normalized exponential function. The fully connected layer maps global features into a four-dimensional score vector. , , , The normalized exponential function transforms the score vector into a probability distribution, with four dimensions corresponding to the probability values ​​of push, rollback, rotation, and stationary operations, respectively. ; in , , , These correspond to four operation types. The sum of all probability values ​​is 1. The value of each dimension in the probability distribution is between 0 and 1. The larger the value, the higher the probability that the classifier judges the current input to belong to that type of operation. The operation type corresponding to the highest probability value is the recognition result of the classifier.

[0059] Determining that the catheter device is under abnormal stress and outputting a warning message specifically includes: The anomaly detection classifier takes the probability distribution sequence output by the operation type classifier at multiple consecutive time steps and the category entropy value of the probability distribution sequence as input, the category entropy value is the Shannon entropy value of the probability distribution at each time step, and the anomaly detection classifier outputs a binary classification of normal operation state and abnormal force state. When the probability of the anomaly classifier outputting a normal operating state is lower than the preset threshold and the category entropy value is higher than the preset entropy value threshold, the catheter device is determined to be in an abnormal stress state. The preset threshold is dynamically adjusted based on the confidence level of the current operation type output by the operation type classifier. The higher the confidence level of the current operation type, the higher the value of the preset threshold.

[0060] Specifically, in one embodiment of the present invention, regarding the determination logic of the anomaly determination classifier, the classifier takes the probability distribution sequence output by the operation type classifier at multiple consecutive time steps and the category entropy value of the probability distribution sequence as input, and takes the normal operation state and the abnormal force state as binary classification output.

[0061] The operation type classifier outputs a four-dimensional probability distribution vector at each time step, where the four dimensions correspond to the probability values ​​of push, rollback, rotation, and stationary operations, respectively. Assume continuous... The probability distribution sequence at each time step is as follows , , , Each of them For a four-dimensional vector: , , , ; in The time step number. Indicates the first Time step The probability value of the class operation, and satisfying: ; The category entropy is the Shannon entropy value of the probability distribution at each time step, used to quantify the degree of disorder in the probability distribution. For the th The entropy of the probability distribution at each time step is calculated as follows: ; When the probability distribution is concentrated in a certain category, for example , , , Entropy A probability distribution approaching 0 indicates high confidence in the classifier's identification result at that time step. When the probability distribution is dispersed across multiple classes, for example... , , , Entropy A large value indicates significant uncertainty in the classifier's recognition result at that time step. Continuous The entropy values ​​at each time step constitute an entropy value sequence. , , , The input features of the anomaly detection classifier are a sequence of probability distributions. , , , With entropy sequence , , , The joint representation of . The dimension of this joint representation is . ,Right now Each time step contains a four-dimensional probability value plus a one-dimensional entropy value, which includes both the output probability information of the classifier at each time step and the degree of disorder in the output probability distribution at each time step.

[0062] The anomaly detection classifier outputs a binary classification result, determining whether the current catheter instrument is in a normal operating state or an abnormal stress state. This classifier outputs the probability value of the normal operating state through a normalized exponential function in the binary classification output layer. Probability value of abnormal stress state And satisfy: ; When the anomaly detection classifier outputs the probability of normal operation status Below the preset threshold And category entropy value Higher than the preset entropy threshold When the catheter is under abnormal stress, it is determined that the catheter device is in an abnormal stress state. The reason for using a dual condition is that relying solely on probability values ​​may lead to misjudgments due to inaccurate classifier calibration, and relying solely on entropy values ​​may lead to false alarms due to the temporary morphological complexity of the catheter when it normally passes through a vessel bifurcation. During normal operation, catheter morphological changes follow defined kinematic laws, and the classifier stably outputs a high-probability single category in consecutive time steps, with the entropy value at each time step... All remained at low levels, and the anomaly detection classifier output... Approaching 1. When the catheter is under abnormal stress, such as abnormal resistance at the tip against the vessel wall or catheter sticking within the vessel lumen, the catheter's morphological movement will be unexpectedly disturbed, and the morphological changes will deviate from the kinematic laws of normal operation. The probability distribution of the classifier output tends to be dispersed across different operation types, and the entropy value at each time step... Increased output of the anomaly detection classifier Decrease. When both conditions are met... Below and Higher than Anomaly detection is triggered when two conditions are met, effectively avoiding false alarms or missed alarms when relying on a single condition.

[0063] Preset threshold The confidence level is dynamically adjusted based on the confidence score of the current operation type output by the operation type classifier. The confidence score of the current operation type is defined as the maximum value of the probability distribution output by the operation type classifier at the current time step. , , , ; when A higher threshold indicates that the operation type classifier has a high degree of confidence in the current recognition result. In this case, the preset threshold will be set. Adjust in the direction of increasing values, that is, require the anomaly classifier to output higher values. Only then will they be allowed to pass, and the criteria for judgment are more stringent; when A low threshold indicates significant uncertainty in the operation type classifier's recognition results; in this case, the preset threshold should be set. The adjustment is made in the direction of decreasing values, with relatively lenient judgment criteria. The specific method for threshold adjustment is as follows: ; in Based on the threshold, This is the preset adjustment coefficient. When hour, Greater than The judgment criteria are stricter; when hour, Less than The judgment criteria are more lenient. The physical basis for the above threshold adjustment method is that when the operation type classifier is highly confident in the current recognition result, its output probability distribution should be stable and sharp. At this time, if the anomaly judgment classifier still outputs a low probability distribution, the judgment will be more lenient. If the error is not due to a true abnormal state rather than the uncertainty of the classifier itself, then a stricter judgment criterion should be adopted to ensure the reliability of the warning. Conversely, when the confidence of the operation type classifier itself is low, the disorder of its output probability distribution is more likely to be due to the ambiguity of the operation type itself. In this case, relaxing the judgment criterion can avoid frequent false alarms due to the uncertainty of the classifier itself.

[0064] In this embodiment, a preset entropy threshold is used. The entropy value can be determined based on the statistical distribution of entropy values ​​under normal operating conditions in historical data. For example, the mean entropy value under normal operating conditions plus twice the standard deviation can be used as the entropy value. .when When the value exceeds this threshold, it indicates that the probability distribution at the current time step is more disordered than the normal fluctuation range, which is consistent with the characteristic of the classifier output probability distribution showing a decrease in class confidence when subjected to abnormal forces.

[0065] The training sample set for constructing the anomaly detection classifier includes: Historical interventional image data sequences and their corresponding stress state labels are obtained, the stress state labels include normal operation state and abnormal stress state; the probability distribution sequence and category entropy value output by the operation type classifier at multiple consecutive time steps are used as feature inputs of training samples, and the corresponding stress state labels are used as training labels to train the anomaly determination classifier.

[0066] Specifically, in one embodiment of the present invention, regarding the training of the anomaly detection classifier, before inputting the probability distribution of the operation type classifier output at multiple consecutive time steps into the pre-trained anomaly detection classifier, it is necessary to construct a training sample set to train the classifier.

[0067] Acquire historical interventional imaging data sequences and their corresponding force state annotations. These sequences are fluoroscopic image records collected from previous interventional surgical cases. Each image sequence covers at least one complete procedure, including the complete motion sequence of the catheter instrument within the blood vessel. Simultaneously, a specialist annotates the force state of each consecutive frame in the historical interventional imaging data sequence, indicating whether the force state corresponds to a normal or abnormal operation. A normal operation refers to the force state where the catheter instrument moves normally within the blood vessel according to the surgeon's intention, and the catheter morphology conforms to normal kinematic laws; for example, the catheter naturally bends along the vessel's direction at a bifurcation, and the tip can pass smoothly. An abnormal force state refers to the force state where the catheter instrument experiences unexpected abnormal resistance or jamming within the blood vessel, and the catheter morphology deviates from normal kinematic laws; for example, the catheter tip presses against the vessel wall and cannot advance, the catheter becomes twisted within the blood vessel, or a sudden increase in resistance during catheter advancement leads to abrupt morphological changes. The labeling method is to assign a binary label to the stress state corresponding to each frame of the image, or to assign the same stress state label to several consecutive frames of the image, so that each segment of image data sequence has a corresponding stress state labeling sequence on the time axis.

[0068] Multi-scale edge detection, category classification, curvature tracking, and curvature morphology change index calculation are performed on historical interventional image data sequences. The specific implementation methods for these processes are completely consistent with the aforementioned processing methods for current interventional images. For each frame in the historical images, multi-scale image transformation is performed, and edge detection is conducted at each scale. Based on pixel width differences, the image region of the catheter instrument is divided into at least two categories, obtaining the image region for each category. For each category, skeleton extraction is performed in each frame to obtain the centerline path. Path points are sampled along the centerline path, and curvature is calculated to generate a time series of curvature. A time series of the curvature morphology change index is calculated based on the time series of curvature. For each category in each frame, the curvature of each sampled point in the current frame is subtracted from the curvature of the corresponding sampled point in the previous frame, and the absolute values ​​are summed to obtain the curvature morphology change index for the current frame. The curvature morphology change indices of multiple consecutive frames are arranged in chronological order to generate a time series of the curvature morphology change index.

[0069] Historical interventional image data sequences are processed using a pre-trained operation type classifier. This operation type classifier, which is the previously trained one, takes as input a time series of the curvature change index and outputs a probability distribution for each operation type. The time series of curvature change indices for each category in the historical image data sequence are input into this operation type classifier to obtain the probability distribution sequence output by the classifier over multiple consecutive time steps. Let a time window contain consecutive... Each time step, for each time step The operation type classifier outputs a four-dimensional probability distribution vector. : , , , in , , , The first The time step corresponds to the probability values ​​of push, rollback, rotation, and standstill operations. The probability distribution sequence corresponding to this time window is as follows: , , , .

[0070] Calculate the category entropy value at each time step in the probability distribution sequence. For the... The entropy of the probability distribution at each time step is calculated as follows: ; in For the first Time step The probability value of the operation. Obtain the entropy sequence corresponding to this time window. , , , .

[0071] The probability distribution sequence corresponding to this time window , , , With entropy sequence , , , The joint representation of is used as the feature input for the training samples. Specifically, the feature vector of each training sample is composed of . It is composed of features from each time step, and each time step contains a four-dimensional probability value. , , , and one-dimensional entropy There are 5 dimensions in total, therefore the total dimension of the feature vector of each training sample is . The training labels are the force state annotations corresponding to the center moment of the time window or most frames within the time window. When the force state annotation is the normal operating state, the training label is... When the force state is labeled as an abnormal force state, the training label is... Training samples are extracted from historical image data sequences using a sliding window approach, with a sliding step size of [value missing]. Starting from the beginning of the time series, take the first to the last position sequentially. Frame, First To the Frame, First To the Frame by frame, and so on, the joint representation of the probability distribution sequence and entropy value sequence within each time window constitutes a feature vector of a training sample, and the force state label corresponding to that time window serves as the training label. The feature vectors of the training samples are associated and stored with their corresponding training labels to construct the training sample set for the anomaly detection classifier. Each training sample is represented as... , ,in For dimension eigenvectors, This is a binary category label.

[0072] All training samples are input into the anomaly classifier to be trained. This classifier can be implemented using a fully connected neural network or a temporal neural network. In one embodiment of the invention, the classifier is a fully connected neural network containing one hidden layer. The input layer receives data in dimension... The feature vector is denoted by , the hidden layer contains a predetermined number of neurons, using modified linear units as the activation function, and the output layer contains two neurons, using a normalized exponential function to output the probability values ​​corresponding to the normal operating state and the abnormal force state. Let the feature vector be denoted by , the hidden layer contains a predetermined number of neurons, the output layer contains two neurons, and the output layer contains two neurons, using a normalized exponential function to output the probability values ​​corresponding to the normal operating state and the abnormal force state. The feature vector of each training sample is Its corresponding real label is The probability of abnormal force state output by the classifier is Then, the binary cross-entropy is used as the loss function: ; in This represents the total number of training samples. The backpropagation algorithm is used to calculate the gradient of the loss function with respect to each network parameter. The gradient descent optimization algorithm is then used to update the network parameters. After multiple rounds of iterative training, training stops when the loss function converges to a preset termination condition or reaches a preset maximum number of iterations. The current network parameters are saved, resulting in a trained anomaly classifier. The trained anomaly classifier can determine anomalies based on continuous input... The system outputs the probability distribution sequence of each time step and the corresponding entropy sequence, and outputs the probability values ​​corresponding to the current catheter device being in normal operation state and abnormal stress state, thereby achieving accurate determination of abnormal stress. The feature input of the training samples also includes the time-series cross-correlation values ​​between the time series of the bending morphology change index of different categories. These cross-correlation values ​​are used as auxiliary features and are input together with the probability distribution sequence and category entropy value into the anomaly classification classifier for training.

[0073] The anomaly detection classifier also takes the time-series cross-correlation value between the time series of the bending morphology change index of different categories as input; When the time-series cross-correlation value is lower than a preset correlation threshold, the preset threshold shifts in the direction of increasing value.

[0074] Specifically, in one embodiment of the present invention, the calculation and determination method for the time-series cross-correlation values ​​between time series of different categories of bending morphology change indices is as follows.

[0075] After dividing the image region of the catheter instrument into at least two categories through the aforementioned steps, a time series of the corresponding tortuosity change index is obtained for each category. Assuming there are two categories, a first category and a second category, the time series of the tortuosity change index for the first category is as follows: , , , The time series of the bending morphology change index for the second category is: , , , ,in Total number of frames Indicates the first The bending morphology change index of the first category of frames. Indicates the first The bending morphology change index of the second category of frames. Both time series are of the same length. And the frame numbers at each time step correspond one-to-one.

[0076] Temporal cross-correlation values ​​are used to quantify the degree of coordinated change between two time series. In interventional procedures, different segments of the same catheter device are interconnected continuous elastic bodies. As the catheter moves within the blood vessel lumen, each segment should undergo coordinated morphological changes. When the distal segment bends to conform to the vascular pathway, its bending morphological change is transmitted along the catheter to the proximal segment, and the two show a synchronous trend in temporal change. Therefore, the temporal cross-correlation values ​​between time series of different categories of bending morphological change indices reflect the coordinated consistency of morphological changes in different segments of the catheter device.

[0077] The time-series cross-correlation value is calculated as follows. First, the time series of the first category of bending morphology change index is calculated. Mean: ; Calculate the time series of the second category of bending morphology change index Mean: ; Calculate the time series of the first category of bending morphology change index Standard deviation: ; Calculate the time series of the second category of bending morphology change index Standard deviation: ; Calculate the Pearson correlation coefficient between the two time series and use it as the time series cross-correlation value. : ; in The value range is from -1 to 1. When When the value is close to 1, it indicates a positive correlation between the two time series. That is, as the bending morphology change index of the first category increases, the bending morphology change index of the second category also increases synchronously. The two exhibit a consistent trend of coordinated change, indicating that the morphological changes in different segments of the catheter have good mechanical transmission consistency. A value close to 0 indicates that the two time series are not significantly correlated, meaning that the morphological changes in the first and second categories are independent and lack synergy, potentially indicating abnormal mechanical transmission within the duct. When the value is close to -1, it indicates that the two time series are negatively correlated. That is, when the first category of bending morphology change index increases, the second category decreases instead. The two change trends are opposite, which usually indicates that the duct has kinked or locally stuck, causing the force transmission direction to reverse.

[0078] The time-series cross-correlation value was obtained through calculation. Subsequently, the anomaly detection classifier uses it as an auxiliary input feature. When the catheter device is under normal stress, the morphological changes of each segment should remain consistent and coordinated. The value is close to 1. When the catheter device is under abnormal stress, such as when the distal segment becomes stuck in the blood vessel lumen while the proximal segment is still being pushed, or when the catheter becomes kinked, causing the force transmission path to be blocked, the morphological changes of each segment will lose their coordination. The value decreased significantly. Specifically, when Below the preset relevant threshold When the anomaly classifier determines that the morphological changes of different segments of the current catheter device lack coordination, the preset threshold is then applied. Shift in the direction of increasing value.

[0079] The preset threshold offset method is as follows. Let the base threshold be... ,when At that time, the preset threshold remains unchanged, that is: ; when At that time, the preset threshold shifts in the direction of increasing value in the following manner: ; in This is a preset offset coefficient used to control the offset magnitude. This is the difference between the relevant threshold and the actual cross-correlation value. A larger difference indicates a more severe lack of synergy in the morphological changes of each segment, and a greater shift towards increasing the value of the preset threshold. Preset Threshold After shifting in the direction of increasing value, the probability of normal operation... A higher threshold is required to maintain normal judgment, making it easier to meet the triggering conditions for anomaly judgment. When the cross-correlation value decreases significantly, it indicates that the morphological changes of different catheter segments have lost their synergy. At this point, even if the probability distribution output by the operation type classifier does not show a significant decrease in confidence, an anomaly warning should still be triggered. This is because, as a continuous elastic body, the deformation of each segment of the catheter should be mechanically coupled, and when this coupling relationship is broken, it itself indicates an abnormal state. By adopting the above offsetting method, the decrease in cross-correlation value can dynamically affect the sensitivity of anomaly judgment, improving the comprehensiveness of anomaly detection.

[0080] In one embodiment of the invention, when three or more categories exist, the temporal cross-correlation values ​​between each pair of categories are calculated separately. Assuming there are three categories, three time series of bending morphological change indices are obtained respectively. , , Calculate separately and Cross-correlation value between , and Cross-correlation value between , and Cross-correlation value between The minimum value among them is taken as the comprehensive cross-correlation value: , , ; The minimum value is compared with a preset correlation threshold. This is done because the loss of synergy in morphological changes between any two segments indicates an abnormality in mechanical transmission within the catheter. Taking the minimum value allows for more sensitive detection of local abnormalities and avoids masking abnormalities between other categories due to a high cross-correlation value between one pair of categories.

[0081] The physical basis for using the aforementioned time-series cross-correlation values ​​in anomaly detection lies in the fact that the same catheter device is physically a continuous whole, and its different segments are subject to the same kinematic constraints and mechanical transmission paths during interventional procedures. When the various parts of the catheter deform in a coordinated manner, the cross-correlation value is high, which conforms to the physical expectation of normal stress. When the cross-correlation value is low, it indicates that the deformation of the various parts of the catheter is not coordinated, which may mean catheter torsion, local entrapment, or uneven stress on different segments. The introduction of this auxiliary criterion makes anomaly detection no longer solely dependent on changes in the probability distribution output by the operation type classifier, but also considers the mechanical coupling relationship between the various segments of the catheter. When the coordination between the two time series decreases, the anomaly detection classifier is more likely to trigger an alert, thereby reducing the false negative rate.

[0082] A catheter device identification system, comprising: The image acquisition module is used to acquire image data sequences of catheter instruments in interventional images, wherein the image data sequences contain multiple consecutive frames of interventional images; The category segmentation module is used to perform multi-scale edge detection on the image data, extract the edge response features of the catheter device at different scales, and divide the image region of the catheter device into at least two categories based on the pixel width difference of the edge response features. The time-series tracking module is used to track the morphological changes of each category in the continuous multi-frame interventional images, extract the curvature of the category in each frame, and generate a time series of curvature. An index calculation module is used to calculate a time series of the bending morphology change index based on the time series of the bending curvature. The operation recognition module has a built-in pre-trained operation type classifier, which is used to input the time series of the bending morphology change index into the operation type classifier and output the probability distribution of the catheter instrument in each operation type. An anomaly detection module is used to input the probability distribution output by the operation type classifier at multiple consecutive time steps into a pre-trained anomaly detection classifier. When the probability of the anomaly detection classifier outputting the normal operation state is lower than a preset threshold, the catheter device is determined to be in an abnormal stress state and an early warning message is output. In the anomaly determination module, when the catheter device is under abnormal stress, its morphological changes deviate from the kinematic laws of normal operation, and the probability distribution output by the operation type classifier at multiple consecutive time steps exhibits a category confidence decay characteristic.

[0083] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0084] In the description of this application, it should be noted that the terms "first", "second", and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0085] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0086] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0087] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0088] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0089] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The protection scope of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the technical scope disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the protection scope of this application. Therefore, the protection scope of this application should be determined by the protection scope of the claims.

[0090] Furthermore, although the operations of the method of this application are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

Claims

1. A method for identifying catheter devices, characterized in that, include: Acquire image data sequences of catheter instruments in interventional imaging, wherein the image data sequences comprise multiple consecutive frames of interventional images; Multi-scale edge detection is performed on the image data to extract the edge response features of the catheter device at different scales. Based on the pixel width difference of the edge response features, the image region of the catheter device is divided into at least two categories. For each category, its morphological changes are tracked in the continuous multi-frame interventional images, the curvature of the category in each frame is extracted, and a time series of curvature is generated. Calculate the time series of the bending morphology change index based on the time series of the bending curvature; The time series of the bending morphology change index is input into a pre-trained operation type classifier, which outputs the probability distribution of the catheter instrument for each operation type. The probability distribution output by the operation type classifier at multiple consecutive time steps is input into a pre-trained anomaly detection classifier. When the probability of the anomaly detection classifier outputting the normal operation state is lower than a preset threshold, the catheter device is determined to be in an abnormal stress state and an early warning message is output. When the catheter device is under abnormal stress, its morphological changes deviate from the kinematic laws of normal operation, and the probability distribution output by the operation type classifier at multiple consecutive time steps exhibits a category confidence decay characteristic.

2. The catheter device identification method according to claim 1, characterized in that, The image region of the catheter device is divided into at least two categories based on the pixel width difference of the edge response features, specifically including: The image data is subjected to multi-scale image transformation, and edge detection is performed at each scale to obtain a multi-scale edge response map; At the same spatial location, the pixel width values ​​of the edge response at different scales are statistically analyzed, and the corresponding catheter instrument image regions at spatial locations where the pixel width values ​​change along the scale direction and meet the preset jump conditions are divided into different categories.

3. The catheter device identification method according to claim 1, characterized in that, The at least two categories include a first category and a second category, wherein the edge response pixel width of the first category is smaller than the edge response pixel width of the second category.

4. The catheter device identification method according to claim 1, characterized in that, Generating a time series of curvature, specifically including: For each category, skeleton extraction is performed in each frame of the interventional image to obtain the centerline path of that category; Multiple path points are sampled at equal intervals along the centerline path, and the curvature at each path point is calculated. The curvature at each of the path points is arranged in order of distance from the centerline path to form the curvature distribution of this category in a single frame image; Arrange the curvature distribution of multiple consecutive frames in time sequence to generate a time series of the bending curvature corresponding to that category.

5. The catheter device identification method according to claim 1, characterized in that, Before inputting the time series of the bending morphology change index into the pre-trained operation type classifier, the method further includes constructing a training sample set for the operation type classifier: Obtain historical interventional image data sequences and their corresponding operation type labels, wherein the operation type labels include push operation, pullback operation, rotation operation and stationary operation; Perform the multi-scale edge detection and category classification on the historical interventional image data sequence to obtain the curvature time series of each category; The time series of the bending morphology change index is calculated based on the bending curvature time series and used as the feature vector of the training samples. The operation type classifier is trained using the corresponding operation type label as the training label.

6. The catheter device identification method according to claim 1, characterized in that, The operation type classifier is a temporal neural network classifier, which is used for: Receive the time-series data of the bending morphology change index; The time series data is subjected to time series feature extraction, and the time series features include the statistics of the bending morphology change index at each time step and the change trend between the bending morphology change index of adjacent time steps. The basic attention weights are assigned based on the distance of each path point from the far end of the category. The closer the path point is to the far end, the higher the basic attention weight. The basic attention weights are then dynamically adjusted based on the curvature change at each path point. The far end is the end of the path that penetrates into the body of the category. The probability distribution of the catheter device for each operation type is output based on the extracted temporal features and the adjusted attention weights.

7. The catheter device identification method according to claim 1, characterized in that, Determining that the catheter device is under abnormal stress and outputting a warning message specifically includes: The anomaly detection classifier takes the probability distribution sequence output by the operation type classifier at multiple consecutive time steps and the category entropy value of the probability distribution sequence as input, the category entropy value is the Shannon entropy value of the probability distribution at each time step, and the anomaly detection classifier outputs a binary classification of normal operation state and abnormal force state. When the probability of the anomaly classifier outputting a normal operating state is lower than the preset threshold and the category entropy value is higher than the preset entropy value threshold, the catheter device is determined to be in an abnormal stress state. The preset threshold is dynamically adjusted based on the confidence level of the current operation type output by the operation type classifier. The higher the confidence level of the current operation type, the higher the value of the preset threshold.

8. The catheter device identification method according to claim 7, characterized in that, The training sample set for constructing the anomaly detection classifier includes: Historical interventional image data sequences and their corresponding stress state labels are obtained, the stress state labels include normal operation state and abnormal stress state; the probability distribution sequence and category entropy value output by the operation type classifier at multiple consecutive time steps are used as feature inputs of training samples, and the corresponding stress state labels are used as training labels to train the anomaly determination classifier.

9. The catheter device identification method according to claim 7, characterized in that, The anomaly detection classifier also takes the time-series cross-correlation value between the time series of the bending morphology change index of different categories as input; When the time-series cross-correlation value is lower than a preset correlation threshold, the preset threshold shifts in the direction of increasing value.

10. A catheter device identification system, characterized in that, For performing the method according to any one of claims 1 to 9, comprising: The image acquisition module is used to acquire image data sequences of catheter instruments in interventional images, wherein the image data sequences contain multiple consecutive frames of interventional images; The category segmentation module is used to perform multi-scale edge detection on the image data, extract the edge response features of the catheter device at different scales, and divide the image region of the catheter device into at least two categories based on the pixel width difference of the edge response features. The time-series tracking module is used to track the morphological changes of each category in the continuous multi-frame interventional images, extract the curvature of the category in each frame, and generate a time series of curvature. An index calculation module is used to calculate a time series of the bending morphology change index based on the time series of the bending curvature. The operation recognition module has a built-in pre-trained operation type classifier, which is used to input the time series of the bending morphology change index into the operation type classifier and output the probability distribution of the catheter instrument in each operation type. An anomaly detection module is used to input the probability distribution output by the operation type classifier at multiple consecutive time steps into a pre-trained anomaly detection classifier. When the probability of the anomaly detection classifier outputting the normal operation state is lower than a preset threshold, the catheter device is determined to be in an abnormal stress state and an early warning message is output. In the anomaly determination module, when the catheter device is under abnormal stress, its morphological changes deviate from the kinematic laws of normal operation, and the probability distribution output by the operation type classifier at multiple consecutive time steps exhibits a category confidence decay characteristic.

Images