An interventional puncture ultrasound-assisted positioning method and system

Abstract

The present application relates to the technical field of puncture needle movement evaluation, in particular to an interventional puncture ultrasound auxiliary positioning method and system, the method comprising: in the interventional puncture process, real-time acquisition of ultrasound images through an ultrasound probe, determination of the optical flow vector corresponding to each pixel point in each ultrasound image; clustering of the pixel points in each ultrasound image; determination of the matching degree between each target cluster and the target cluster in the next frame of ultrasound image of the ultrasound image to which the target cluster belongs; screening of the target clusters matched with each other from all ultrasound images; determination of the possible index of the puncture needle corresponding to each target cluster sequence; screening of the puncture needle region sequence and the needle tip position from each target cluster in the puncture needle region sequence; evaluation of the needle tip movement direction and needle tip displacement of the puncture needle at the next moment. The present application realizes interventional puncture ultrasound positioning tracking and improves the accuracy of the evaluation and positioning of the next moment movement of the puncture needle.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of puncture needle movement evaluation, in particular to an interventional puncture ultrasound auxiliary positioning method and system. BACKGROUND

[0002] Interventional puncture is a medical operation method of puncturing, inserting instruments into the human body through the skin, mucosa or other tissues for diagnosis or treatment. In the process of interventional puncture, the position of the puncture needle at the current moment is often determined by the doctor observing the ultrasound image collected at the current moment, so as to evaluate the movement of the puncture needle at the next moment, and then realize the positioning tracking of the puncture needle.

[0003] However, when the movement of the puncture needle at the next moment is evaluated by the doctor observing the ultrasound image collected at the current moment, the following technical problems often exist:

[0004] In the process of interventional puncture, due to various reasons such as poor contact of the ultrasound probe, it is often difficult to ensure that all the collected ultrasound images are clear. Therefore, if the ultrasound image collected at the current moment is not clear, the doctor often has difficulty in accurately evaluating the situation of the puncture needle at the next moment through the ultrasound image collected at the current moment, thereby resulting in poor accuracy of evaluating the movement of the puncture needle at the next moment, and poor accuracy of interventional puncture ultrasound positioning tracking. SUMMARY

[0005] In order to solve the technical problem of poor accuracy of interventional puncture ultrasound positioning tracking caused by poor accuracy of evaluating the movement of the puncture needle at the next moment, the present application provides an interventional puncture ultrasound auxiliary positioning method and system.

[0006] In the first aspect, the present application provides an interventional puncture ultrasound auxiliary positioning method, which comprises:

[0007] In the process of interventional puncture, the ultrasound image is collected in real time by the ultrasound probe, and the optical flow vector corresponding to each pixel point in each ultrasound image is determined according to the motion between each ultrasound image and the next frame of ultrasound image thereof;

[0008] According to the similarity between the optical flow vectors corresponding to the pixel points in each ultrasound image and the distance between the pixel points, the pixel points in each ultrasound image are clustered to obtain target clusters;

[0009] According to the distribution of non-overlapping pixel points between each target cluster and the target cluster in the next frame of ultrasound image of the ultrasound image to which the target cluster belongs, and the center of each target cluster and the optical flow vector corresponding to the pixel points in each target cluster, the matching degree between each target cluster and the target cluster in the next frame of ultrasound image of the ultrasound image to which the target cluster belongs is determined;

[0010] screening the target clusters matched with each other from all the ultrasound images to form a target cluster sequence based on the matching degree between the target clusters;

[0011] determining a possible needle index corresponding to each target cluster sequence according to the matching degree between adjacent target clusters in each target cluster sequence, the similarity of the optical flow vectors between the adjacent target clusters, and the shape feature of the target cluster;

[0012] screening a needle region sequence from all the target cluster sequences based on all the possible needle indexes, and screening a needle tip position from each target cluster in the needle region sequence;

[0013] evaluating the movement direction and displacement of the needle tip of the next moment of the puncture needle according to the distance between the needle tip positions in adjacent needle regions in the needle region sequence and the optical flow vectors corresponding to the pixel points in the needle region.

[0014] In a possible implementation manner of the first aspect, the clustering the pixel points in each frame of ultrasound image according to the similarity between the optical flow vectors corresponding to the pixel points and the distance between the pixel points to obtain the target cluster comprises:

[0015] determining a normalized value of the cosine similarity between the optical flow vectors corresponding to each two pixel points in each frame of ultrasound image as a vector similarity factor between each two pixel points in each frame of ultrasound image;

[0016] determining an absolute value of the difference between the modules of the optical flow vectors corresponding to each two pixel points in each frame of ultrasound image as a motion distance factor between each two pixel points in each frame of ultrasound image;

[0017] determining a consistency coefficient between each two pixel points in each frame of ultrasound image according to the vector similarity factor, the motion distance factor and the Euclidean distance between each two pixel points in each frame of ultrasound image, wherein the vector similarity factor is positively correlated with the consistency coefficient, and the motion distance factor and the Euclidean distance are negatively correlated with the consistency coefficient;

[0018] clustering the pixel points in each frame of ultrasound image according to the consistency coefficients between all the pixel points in each frame of ultrasound image, and recording each obtained clustering cluster as a target cluster.

[0019] In a possible implementation manner of the first aspect, the determining the matching degree between each target cluster and the target cluster in the next frame of ultrasound image of the ultrasound image to which the target cluster belongs according to the distribution of the non-overlapping pixel points between the target cluster and the target cluster in the next frame of ultrasound image of the ultrasound image to which the target cluster belongs, and the center of each target cluster and the optical flow vectors corresponding to the pixel points in each target cluster comprises:

[0020] determining any one target cluster in a next frame of ultrasound image to which the first marked cluster belongs as a second marked cluster;

[0021] determining a region in the next frame of ultrasound image to which the first marked cluster belongs and which has the same position as the first marked cluster as a reference region, and determining a center of the reference region as a reference center;

[0022] determining an intersection of the second marked cluster and the reference region as a reference intersection region, and determining a center of a region composed of all pixel points in the second marked cluster except the reference intersection region as a candidate center, wherein the pixel points in the second marked cluster except the reference intersection region represent non-overlapping pixel points between the first marked cluster and the second marked cluster;

[0023] determining a direction in which the reference center points to the candidate center as a reference motion direction;

[0024] determining a mean value of optical flow vectors corresponding to all pixel points in the first marked cluster as a candidate motion vector, and determining a direction of the candidate motion vector as a candidate motion direction;

[0025] determining a matching degree between the first marked cluster and the second marked cluster according to an included angle between the reference motion direction and the candidate motion direction, an Euclidean distance between the reference center and a center of the second marked cluster, and a number of pixel points in the reference intersection region.

[0026] In a possible implementation manner of the first aspect, the matching degree between the first marked cluster and the second marked cluster corresponds to a formula as follows:

[0027] wherein D is the matching degree between the first marked cluster and the second marked cluster; is a normalization function; n is the number of pixel points in the reference intersection region; is a natural exponential function; is the included angle between the reference motion direction and the candidate motion direction; d is the Euclidean distance between the reference center and the center of the second marked cluster.

[0028] In a possible implementation manner of the first aspect, the determining of the possible index of the puncture needle corresponding to each target cluster sequence according to the matching degree between adjacent target clusters in each target cluster sequence, the similarity of optical flow vectors between adjacent target clusters, and the shape feature of the target cluster comprises:

[0029] determine a direction consistency corresponding to each target cluster sequence according to a matching degree between adjacent target clusters in each target cluster sequence and a difference between directions of optical flow vectors between the adjacent target clusters;

[0030] determine a deviation distance corresponding to each pixel point in each target cluster as a Euclidean distance between the pixel point and a center of the target cluster to which the pixel point belongs;

[0031] determine a shape feature index corresponding to each target cluster as a variance of the deviation distances corresponding to all pixel points in the target cluster;

[0032] determine a puncture needle possibility index corresponding to each target cluster sequence according to the direction consistency corresponding to each target cluster sequence and the shape feature indexes corresponding to all target clusters in each target cluster sequence, wherein the direction consistency and the shape feature index are in positive correlation with the puncture needle possibility index.

[0033] In a possible implementation manner of the first aspect, the direction consistency corresponding to the target cluster sequence corresponds to a formula as follows:

[0034] ; wherein, is the direction consistency corresponding to the rth target cluster sequence; r is a serial number of the target cluster sequence; is a normalization function; is a number of target clusters in the rth target cluster sequence; t is a serial number of a target cluster in the rth target cluster sequence; is a matching degree between the tth target cluster and the (t+1)th target cluster in the rth target cluster sequence; is a cosine similarity between a motion representative vector corresponding to the tth target cluster and a motion representative vector corresponding to the (t+1)th target cluster in the rth target cluster sequence; the motion representative vector corresponding to the target cluster is a mean value of optical flow vectors corresponding to all pixel points in the target cluster.

[0035] In a possible implementation manner of the first aspect, the determining of the puncture needle possibility index corresponding to each target cluster sequence according to the direction consistency corresponding to each target cluster sequence and the shape feature indexes corresponding to all target clusters in each target cluster sequence comprises:

[0036] determine a puncture needle shape possibility factor corresponding to each target cluster sequence as an accumulated value of the shape feature indexes corresponding to all target clusters in each target cluster sequence;

[0037] normalize a product of the direction consistency and the puncture needle shape possibility factor corresponding to each target cluster sequence to obtain the puncture needle possibility index corresponding to each target cluster sequence.

[0038] In conjunction with the first aspect above, in one possible implementation, the step of selecting the needle tip location from each target cluster in the puncture needle region sequence includes:

[0039] Any target cluster in the puncture needle region sequence is identified as the marked region. A marking ray is constructed with the center of the marked region as the starting point and the direction of the mean of the optical flow vectors corresponding to all pixels in the marked region as the extension direction.

[0040] The pixel furthest from the center of the marking region is selected from the intersection of the marking ray and the marking region and designated as the needle tip pixel. The position of the needle tip pixel is then determined as the needle tip position.

[0041] In conjunction with the first aspect mentioned above, in one possible implementation, the formulas for the direction of needle tip movement and the needle tip displacement at the next moment are as follows:

[0042] ;

[0043] Where R is the direction of the predicted needle tip movement in the next moment; L is the predicted needle tip displacement in the next moment; M is the number of puncture needle regions in the puncture needle region sequence; m is the sequence number of the puncture needle regions in the puncture needle region sequence; Q is the cumulative value of the sequence numbers of all puncture needle regions in the puncture needle region sequence. It is the mean of the optical flow vectors corresponding to all pixels in the m-th puncture needle region sequence; It is the distance between the needle tip position in the m-th puncture needle region and the needle tip position in the (m+1)-th puncture needle region sequence.

[0044] Secondly, the present invention provides an interventional puncture ultrasound-assisted positioning system, the system comprising:

[0045] The image acquisition vector determination module is used to acquire ultrasound images in real time through an ultrasound probe during interventional puncture, and determine the optical flow vector corresponding to each pixel in each ultrasound image based on the motion between each frame of ultrasound image and the next frame of ultrasound image.

[0046] The pixel clustering module is used to cluster the pixels in each frame of ultrasound image according to the similarity between the optical flow vectors corresponding to the pixels and the distance between the pixels, so as to obtain the target cluster.

[0047] The matching degree determination module is configured to determine the matching degree between each target cluster and the target cluster in the next frame of ultrasound images of the ultrasound images to which the target cluster belongs according to the distribution of non-overlapping pixel points between each target cluster and the target cluster in the next frame of ultrasound images of the ultrasound images to which the target cluster belongs, and the center of each target cluster and the optical flow vector corresponding to the pixel points in each target cluster.

[0048] The target cluster screening module is configured to screen the target clusters that match each other from all the ultrasound images to form the target cluster sequence based on the matching degrees between the target clusters.

[0049] The puncture needle possibility index determination module is configured to determine the puncture needle possibility index corresponding to each target cluster sequence according to the matching degrees between adjacent target clusters in each target cluster sequence, the similarity of the optical flow vectors between the adjacent target clusters, and the shape features of the target clusters.

[0050] The needle tip position screening module is configured to screen the puncture needle region sequence from all the target cluster sequences based on all the puncture needle possibility indexes, and screen the needle tip position from each target cluster in the puncture needle region sequence.

[0051] The needle tip motion direction displacement evaluation module is configured to evaluate the needle tip motion direction and the needle tip displacement of the puncture needle at the next moment according to the distance between the needle tip positions in adjacent puncture needle regions in the puncture needle region sequence and the optical flow vectors corresponding to the pixel points in the puncture needle regions.

[0052] In a third aspect, a server is provided, including a memory and a processor. The memory is configured to store executable program code, and the processor is configured to call and run the executable program code from the memory, so that the device executes the method in the first aspect or any possible implementation manner of the first aspect.

[0053] In a fourth aspect, a computer program product is provided, which includes computer program code. When the computer program code is run on a computer, the computer program code causes the computer to execute the method in the first aspect or any possible implementation manner of the first aspect.

[0054] In a fifth aspect, a computer readable storage medium is provided, which stores computer program code. When the computer program code is run on a computer, the computer program code causes the computer to execute the method in the first aspect or any possible implementation manner of the first aspect.

[0055] The present application has the following beneficial effects:

[0056] The intervention puncture ultrasound auxiliary positioning method of the application realizes intervention puncture ultrasound positioning tracking, solves the technical problem of poor accuracy of evaluating the next moment movement of the puncture needle, and improves the accuracy of evaluating the next moment movement of the puncture needle. Compared with evaluating the next moment movement of the puncture needle by a doctor observing the ultrasound image collected at the current moment, when evaluating the next moment movement of the puncture needle, the application comprehensively considers multiple frames of ultrasound images, to a certain extent, reduces the adverse effects of being difficult to accurately position the puncture needle due to the unclarity of a frame of ultrasound image, and quantifies the matching degree and the possible index of the puncture needle, so as to realize the positioning of the needle tip position, and then relatively objectively realize the evaluation of the needle tip movement direction and the needle tip displacement of the puncture needle at the next moment, thereby improving the accuracy of evaluating the next moment movement of the puncture needle, and further improving the accuracy of intervention puncture ultrasound positioning tracking. BRIEF DESCRIPTION OF DRAWINGS

[0057] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0058] Figure 1 The flow chart of the intervention puncture ultrasound auxiliary positioning method of the application;

[0059] Figure 2 The composition structure schematic diagram of the intervention puncture ultrasound auxiliary positioning system of the application;

[0060] Figure 3 The structure schematic diagram of the computer device of the application. DETAILED DESCRIPTION

[0061] In order to further illustrate the technical means and effects adopted by the present application to achieve the predetermined object, the specific implementation, structure, features and effects of the technical solutions proposed by the present application will be described in detail below in combination with the drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures or characteristics in one or more embodiments can be combined in any suitable form.

[0062] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs.

[0063] REFERENCE Figure 1, shows the flow of some embodiments of an interventional puncture ultrasound assisted positioning method of the present application. The interventional puncture ultrasound assisted positioning method comprises the following steps:

[0064] Step S1, in the interventional puncture process, real-time acquisition of ultrasound images is performed by an ultrasound probe, and the optical flow vector corresponding to each pixel point in each ultrasound image is determined according to the motion between each ultrasound image and the next ultrasound image thereof.

[0065] Wherein, the ultrasound probe is a device for emitting and receiving ultrasonic waves in the ultrasonic detection process, which can be used to acquire ultrasound images. The optical flow vector refers to the motion vector of a pixel point in time between two consecutive images, usually represented as a vector.

[0066] It should be noted that interventional puncture is a medical operation method for puncturing, inserting instruments into the human body through the skin, mucosa or other tissues for diagnosis or treatment, which is usually used for positioning, sampling, treating lesions or guiding other medical devices into the target site, and has the advantages of small trauma, rapid recovery, etc. This technology is widely used in the fields of tumors, internal diseases, vascular lesions, etc. Ultrasound image guidance is a common external guidance means for interventional puncture. Before interventional puncture, the target area is located and marked using ultrasound images to plan the puncture route, and then the puncture needle is guided into the target area for sampling or drug injection, etc. The real-time imaging function of ultrasound can help medical staff accurately locate the target area during operation and avoid damaging the surrounding important structures.

[0067] As an example, the ultrasound probe can be placed at a suitable position on the surface of the patient's skin by the doctor's experience, and during the interventional puncture process, real-time acquisition of ultrasound images is performed by the ultrasound probe, that is, multiple ultrasound images can have been acquired before the current time. Then, the LK (Lucas-Kanade) optical flow algorithm can be used to compare the motion between each pixel point in each ultrasound image and the corresponding pixel point in the next ultrasound image, to obtain the optical flow vector corresponding to each pixel point in each ultrasound image.

[0068] It should be noted that the ultrasound images involved in subsequent calculations can be all ultrasound images acquired in real time during the interventional puncture process. In order to improve the calculation efficiency, the ultrasound images involved in subsequent calculations can also be the last acquired preset number of ultrasound images. Wherein, the preset number can be a pre-set number, which can be 100.

[0069] Step S2, according to the similarity between the optical flow vectors corresponding to the pixel points in each ultrasound image, and the distance between the pixel points, the pixel points in each ultrasound image are clustered to obtain a target cluster.

[0070] It should be noted that, in the same frame of image, the motion direction distribution of the pixel points belonging to the same structure often has certain regularity. For the structure inside the human body that moves, such as the part of the human tissue that moves in peristalsis, contraction and expansion, etc., the motion direction of the pixel points in the same structure often does not have a big difference when these tissues move, that is, the motion direction of the pixel points belonging to the same structure is often similar. For the puncture needle which is a straight object with certain hardness, the motion direction and motion distance of all the pixel points on the needle are often consistent when the needle is inserted. Therefore, clustering the motion direction of the pixel points and the distance between the pixel points in the same frame of image can divide the pixel points belonging to different tissue structures. The direction of the optical flow vector corresponding to the pixel point can represent the motion direction of the pixel point to a certain extent. The modulus of the optical flow vector corresponding to the pixel point can represent the motion distance of the pixel point to a certain extent. A target cluster can represent a tissue structure or a puncture needle.

[0071] As an example, the present step can include the following steps:

[0072] Firstly, the normalized value of the cosine similarity between the optical flow vectors corresponding to each two pixel points in each frame of ultrasound image is determined as the vector similarity factor between each two pixel points in each frame of ultrasound image.

[0073] Secondly, the absolute value of the difference between the modulus of the optical flow vectors corresponding to each two pixel points in each frame of ultrasound image is determined as the motion distance factor between each two pixel points in each frame of ultrasound image.

[0074] Thirdly, the consistency coefficient between each two pixel points in each frame of ultrasound image is determined according to the vector similarity factor, the motion distance factor and the Euclidean distance between each two pixel points in each frame of ultrasound image.

[0075] The vector similarity factor can have a positive correlation with the consistency coefficient. The motion distance factor and the Euclidean distance can have a negative correlation with the consistency coefficient.

[0076] For example, the formula for determining the consistency coefficient between two pixel points in the same frame of ultrasound image can be:

[0077] ;

[0078] ; wherein, is the consistency coefficient between the i-th pixel point and the j-th pixel point in the f-th frame of ultrasound image. f is the frame number of the ultrasound image. i and j are the serial numbers of different pixel points in the f-th frame of ultrasound image. is a normalization function. is a vector similarity factor between the i-th pixel point and the j-th pixel point in the f-th ultrasound image. is a cosine similarity between the optical flow vector corresponding to the i-th pixel point and the optical flow vector corresponding to the j-th pixel point in the f-th ultrasound image. is a natural exponential function. is an Euclidean distance between the i-th pixel point and the j-th pixel point in the f-th ultrasound image. is an absolute value function. is a modulus of the optical flow vector corresponding to the j-th pixel point in the f-th ultrasound image. is a modulus of the optical flow vector corresponding to the i-th pixel point in the f-th ultrasound image. is a motion distance factor between the i-th pixel point and the j-th pixel point in the f-th ultrasound image.

[0079] It should be noted that in actual situations, the smaller the distance between the pixel points in the same frame of ultrasound image, the more likely these pixel points belong to the same tissue structure or puncture needle region. When the smaller the Euclidean distance between the i-th pixel point and the j-th pixel point, the more likely the i-th pixel point and the j-th pixel point belong to the same tissue structure or puncture needle region. In actual situations, the motion direction and motion distance of all pixel points on the puncture needle during the puncture process are often consistent. When the more similar the optical flow vector corresponding to the i-th pixel point and the optical flow vector corresponding to the j-th pixel point, the more similar the motion direction and motion distance between the i-th pixel point and the j-th pixel point, the more likely the i-th pixel point and the j-th pixel point belong to the same tissue structure or puncture needle region. When the more similar the motion distance between the i-th pixel point and the j-th pixel point, the more likely the i-th pixel point and the j-th pixel point belong to the puncture needle region. Therefore, when the more likely the i-th pixel point and the j-th pixel point belong to the same tissue structure or puncture needle region.

[0080] Fourthly, according to the consistency factor between all pixel points in each frame of ultrasound image, the pixel points in each frame of ultrasound image are clustered, and each obtained cluster is recorded as a target cluster.

[0081] For example, the pixel points between which the consistency factor in the same frame of ultrasound image is greater than a preset consistency threshold can be constructed into the same cluster, which is recorded as a target cluster. The preset consistency threshold can be a threshold set in advance, which can be 0.7.

[0082] Step S3, determining the matching degree between each target cluster and the target cluster in the next frame of ultrasound image of the ultrasound image to which the target cluster belongs according to the distribution of non-overlapping pixel points between each target cluster and the target cluster in the next frame of ultrasound image of the ultrasound image to which the target cluster belongs, and the center of each target cluster and the optical flow vector corresponding to the pixel points in each target cluster.

[0083] As an example, the present step can include the following steps:

[0084] Firstly, any one target cluster is determined as a first marker cluster, and any one target cluster in the next frame of ultrasound image of the ultrasound image to which the first marker cluster belongs is determined as a second marker cluster.

[0085] Secondly, the region in the next frame of ultrasound image of the ultrasound image to which the first marker cluster belongs and having the same position as the first marker cluster is determined as a reference region, and the center of the reference region is determined as a reference center.

[0086] Thirdly, the intersection of the second marker cluster and the reference region is determined as a reference intersection region, and the center of the region composed of all pixel points in the second marker cluster except the reference intersection region is determined as a candidate center.

[0087] Herein, the pixel points in the second marker cluster except the reference intersection region can represent the non-overlapping pixel points between the first marker cluster and the second marker cluster.

[0088] Fourthly, the direction from the reference center to the candidate center is determined as a reference motion direction.

[0089] It should be noted that the reference motion direction can represent the motion direction between the first marker cluster and the second marker cluster to some extent.

[0090] Fifthly, the mean value of the optical flow vectors corresponding to all pixel points in the first marker cluster is determined as a candidate motion vector, and the direction of the candidate motion vector is determined as a candidate motion direction.

[0091] It should be noted that the candidate motion direction can represent the motion direction of the first marker cluster.

[0092] Sixthly, the matching degree between the first marker cluster and the second marker cluster is determined according to the included angle between the reference motion direction and the candidate motion direction, the Euclidean distance between the reference center and the center of the second marker cluster, and the number of pixel points in the reference intersection region.

[0093] For example, the formula for determining the matching degree between the first marker cluster and the second marker cluster can be as follows:

[0094] ; wherein, D is a matching degree between the first marker cluster and the second marker cluster. is a normalized function. n is a number of pixels in the reference intersection region. is a natural exponential function. is an included angle between the reference motion direction and the candidate motion direction. d is an Euclidean distance between the reference center and the center of the second marker cluster.

[0095] It should be noted that for the same tissue structure in the human body, the position is often relatively fixed, therefore, the same tissue structure is often located at the same position in different frames of ultrasound images. In the puncture process, the moving speed of the puncture needle is relatively slow, and the time length between adjacent frames of ultrasound images is often short, therefore, the position of the puncture needle in adjacent frames of ultrasound images is often relatively close. When n is larger, it often means that there are more pixels at the same position in the first marker cluster and the second marker cluster, it often means that the positions of the first marker cluster and the second marker cluster in the corresponding ultrasound images are closer, and it often means that the first marker cluster and the second marker cluster are more likely to represent the same tissue structure or the puncture needle. When d is smaller, it often means that the positions of the first marker cluster and the second marker cluster in the corresponding ultrasound images are closer, and it often means that the first marker cluster and the second marker cluster are more likely to represent the same tissue structure or the puncture needle. When is smaller, it often means that the motion direction between the first marker cluster and the second marker cluster is more likely to be equal to the motion direction of the first marker cluster, it often means that the second marker cluster is more likely to be the first marker cluster after motion, and it often means that the first marker cluster and the second marker cluster are more likely to represent the same tissue structure or the puncture needle. Therefore, when D is larger, it often means that the first marker cluster and the second marker cluster are more likely to represent the same tissue structure or the puncture needle.

[0096] Step S4, based on the matching degree between the target clusters, the target clusters that match each other are screened out from all the ultrasound images to form a target cluster sequence.

[0097] The target cluster sequence can include: regions corresponding to the same tissue structure or the puncture needle in different ultrasound images. The number of target cluster sequences can be equal to the number of target clusters in each frame of ultrasound image. The target cluster is earlier in the target cluster sequence, the earlier the acquisition time corresponding to the ultrasound image is.

[0098] As an example, if there are 3 frames of ultrasound images, any one target cluster in the first frame of ultrasound image can be determined as a first temporary cluster; the target cluster with the largest matching degree with the first temporary cluster is screened out from the second frame of ultrasound image, and is determined as a second temporary cluster; the target cluster with the largest matching degree with the second temporary cluster is screened out from the third frame of ultrasound image, and is determined as a third temporary cluster; the first temporary cluster, the second temporary cluster and the third temporary cluster can form a target cluster sequence.

[0099] Step S5, according to the matching degree between adjacent target clusters in each target cluster sequence, the light flow vector similarity between adjacent target clusters, and the shape feature of the target cluster, determine the possible needle direction corresponding to each target cluster sequence.

[0100] As an example, the present step can include the following steps:

[0101] First, according to the matching degree between adjacent target clusters in each target cluster sequence, and the light flow vector similarity between adjacent target clusters, determine the direction consistency corresponding to each target cluster sequence.

[0102] For example, the formula for determining the direction consistency corresponding to the target cluster sequence can be:

[0103] ; wherein, is the direction consistency corresponding to the rth target cluster sequence. r is the serial number of the target cluster sequence. is a normalization function. is the number of target clusters in the rth target cluster sequence. t is the serial number of the target cluster in the rth target cluster sequence. is the matching degree between the tth target cluster and the t+1th target cluster in the rth target cluster sequence. is the cosine similarity between the motion representative vector corresponding to the tth target cluster and the motion representative vector corresponding to the t+1th target cluster in the rth target cluster sequence. The motion representative vector corresponding to the target cluster can be the mean of the light flow vectors corresponding to all pixel points in the target cluster.

[0104] It should be noted that, can be used as weight, and is larger, the more likely the calculated has reference significance. In actual situations, since the puncture position and angle of the puncture needle are usually determined in the initial planning of the interventional puncture surgery, the direction of the puncture needle may be adjusted slightly during the interventional puncture surgery to avoid dangerous areas such as blood vessels and nerves. The puncture needle itself is a straight object with a certain degree of hardness, and according to the principle of the surgery and the restrictions of actual operation, the movement direction of the puncture needle usually does not change too much during the entire interventional surgery. When is larger, the more similar the motion representative vector corresponding to the tth target cluster and the motion representative vector corresponding to the t+1th target cluster, the more similar the motion direction between the tth target cluster and the t+1th target cluster, and the more consistent the motion between the tth target cluster and the t+1th target cluster with the motion of the puncture needle. Therefore, when The greater the shape feature index corresponding to the target cluster is, the more likely the rth target cluster sequence represents the puncture needle.

[0105] The second step is to determine the Euclidean distance between each pixel point in each target cluster and the center of the target cluster to which the pixel point belongs as the deviation distance corresponding to each pixel point in each target cluster.

[0106] The third step is to determine the variance of the deviation distances corresponding to all pixel points in each target cluster as the shape feature index corresponding to each target cluster.

[0107] It should be noted that since the puncture needle is a straight object with a certain hardness, the distance distribution between all pixel points on the puncture needle and the center point is relatively discrete. Therefore, when the shape feature index corresponding to the target cluster is greater, the distance distribution between all pixel points in the target cluster and the center point is relatively more discrete, and the target cluster is more likely to represent the puncture needle region.

[0108] The fourth step is to determine the puncture needle possibility index corresponding to each target cluster sequence according to the direction consistency corresponding to each target cluster sequence and the shape feature index corresponding to all target clusters in each target cluster sequence.

[0109] The direction consistency and the shape feature index can be positively correlated with the puncture needle possibility index.

[0110] For example, determining the puncture needle possibility index corresponding to each target cluster sequence can include the following sub-steps:

[0111] The first sub-step is to determine the cumulative value of the shape feature index corresponding to all target clusters in each target cluster sequence as the puncture needle shape possibility factor corresponding to each target cluster sequence.

[0112] The second sub-step is to normalize the product of the direction consistency and the puncture needle shape possibility factor corresponding to each target cluster sequence to obtain the puncture needle possibility index corresponding to each target cluster sequence.

[0113] For example, the formula for determining the puncture needle possibility index corresponding to the target cluster sequence can be:

[0114] ; wherein, is the puncture needle possibility index corresponding to the rth target cluster sequence. r is the serial number of the target cluster sequence. is a normalization function. is the direction consistency corresponding to the rth target cluster sequence. is the puncture needle shape possibility factor corresponding to the rth target cluster sequence.

[0115] It should be noted that when The greater the value is, the more likely the rth target cluster sequence represents the puncture needle. The greater the value is, the more likely the rth target cluster sequence represents the puncture needle. The greater the value is, the more likely the rth target cluster sequence represents the puncture needle.

[0116] Step S6, based on all puncture needle possible indicators, filtering out puncture needle region sequences from all target cluster sequences, and filtering out needle tip positions from each target cluster in the puncture needle region sequences.

[0117] As an example, the step can include the following steps:

[0118] Firstly, filtering out a target cluster sequence corresponding to the maximum puncture needle possible indicator from all target cluster sequences as a puncture needle region sequence.

[0119] Secondly, determining any target cluster in the puncture needle region sequence as a marker region, and constructing a marker ray with the center of the marker region as the starting point and the direction of the mean value of the optical flow vectors corresponding to all the pixels in the marker region as the extension direction.

[0120] Thirdly, filtering out a pixel point farthest from the center of the marker region from the intersection of the marker ray and the marker region as a needle tip pixel point, and determining the position of the needle tip pixel point as the needle tip position.

[0121] It should be noted that in actual situations, the unclear ultrasound images collected due to poor contact of the ultrasound probe and the like are relatively small, and when evaluating the needle tip movement direction and needle tip displacement of the puncture needle at the next moment, the embodiments of the present application not only consider the ultrasound images collected at the current moment, but also consider the ultrasound images collected at multiple historical moments, that is, considering more ultrasound images at the same time can reduce the adverse effects caused by unclear single-frame ultrasound images to a certain extent, therefore, the subsequent evaluation of the needle tip movement direction and needle tip displacement of the puncture needle at the next moment based on the distance between the needle tip positions in adjacent puncture needle regions in the puncture needle region sequence and the optical flow vectors corresponding to the pixel points in the puncture needle region can improve the accuracy of the evaluation of the next movement of the puncture needle to a certain extent.

[0122] Step S7, according to the distance between the needle tip positions in adjacent puncture needle regions in the puncture needle region sequence and the optical flow vectors corresponding to the pixel points in the puncture needle region, evaluating the needle tip movement direction and needle tip displacement of the puncture needle at the next moment.

[0123] It should be noted that the evaluation of the needle tip movement direction and the needle tip displacement of the next moment realizes the positioning of the next moment of the puncture needle, thereby realizing the positioning tracking of the puncture needle, so as to assist the doctor in judging the next moment position of the puncture needle, thereby facilitating the doctor to judge whether the puncture needle movement needs to be fine-tuned.

[0124] As an example, the formula corresponding to the evaluation of the needle tip movement direction and the needle tip displacement of the next moment of the puncture needle can be:

[0125] ;

[0126] ; wherein R is the estimated needle tip movement direction of the next moment of the puncture needle. L is the estimated needle tip displacement of the next moment of the puncture needle. M is the number of puncture needle regions in the puncture needle region sequence. m is the serial number of the puncture needle region in the puncture needle region sequence. Q is the cumulative value of the serial numbers of all puncture needle regions in the puncture needle region sequence. is the mean value of the optical flow vectors corresponding to all pixel points in the mth puncture needle region in the puncture needle region sequence. is the distance between the needle tip position in the mth puncture needle region and the needle tip position in the m+1th puncture needle region in the puncture needle region sequence. The method for obtaining the distance between the needle tip position in the mth puncture needle region and the needle tip position in the m+1th puncture needle region can be: selecting the same position as the needle tip position in the mth puncture needle region from the ultrasound image to which the m+1th puncture needle region belongs, denoted as the reference position, and determining the Euclidean distance between the needle tip position in the m+1th puncture needle region and the reference position as the distance between the needle tip position in the mth puncture needle region and the needle tip position in the m+1th puncture needle region.

[0127] It should be noted that in actual situations, the puncture needle often moves continuously during the puncture process, and its previous movement often affects its subsequent movement, so the movement of the puncture needle in the collected ultrasound image can be used to estimate the movement of the puncture needle at the next moment, and the movement of the puncture needle at the next moment can reflect the position reached by the puncture needle at the next moment to a certain extent. When m is smaller, it means that the collection time corresponding to the ultrasound image to which the puncture needle region belongs is earlier, and the influence degree of the next moment movement is smaller. The value range of is (0, 1), which can be used as the weight of and and . The motion of the mth puncture needle region can be characterized, and the greater the value is, the greater the motion degree of the puncture needle in the ultrasound image to which the mth puncture needle region belongs is, and the greater the motion degree of the puncture needle in the next moment is. Therefore, the direction of R can represent the motion direction of the needle tip of the puncture needle to a certain extent. When the greater the value is, the greater the distance between the needle tip position in the mth puncture needle region and the needle tip position in the mth+1 puncture needle region is, the greater the motion displacement of the puncture needle between the adjacent two frames of ultrasound images is, and the greater the motion displacement of the puncture needle in the next moment is. Therefore, L can represent the motion displacement of the needle tip of the puncture needle to a certain extent.

[0128] Reference Figure 2 Based on the same inventive concept as the above method embodiments, the present application provides an interventional puncture ultrasound auxiliary positioning system, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the computer program implements the steps of an interventional puncture ultrasound auxiliary positioning method when executed by the processor, and can specifically comprise:

[0129] An image acquisition vector determination module 201 is configured to acquire ultrasound images in real time through an ultrasound probe during an interventional puncture process, and determine a flow vector corresponding to each pixel point in each ultrasound image according to the motion between each ultrasound image and the next ultrasound image thereof.

[0130] A pixel clustering module 202 is configured to cluster the pixel points in each ultrasound image according to the similarity between the flow vectors corresponding to the pixel points and the distance between the pixel points, to obtain target clusters.

[0131] A matching degree determination module 203 is configured to determine the matching degree between each target cluster and the target cluster in the next ultrasound image of the ultrasound image to which the target cluster belongs according to the distribution of non-overlapping pixel points between the target cluster and the target cluster in the next ultrasound image, and the center of each target cluster and the flow vector corresponding to each pixel point in each target cluster.

[0132] A target cluster screening module 204 is configured to screen the target clusters that match each other from all ultrasound images based on the matching degree between the target clusters, to form a target cluster sequence.

[0133] A puncture needle possible indicator determination module 205 is configured to determine a puncture needle possible indicator corresponding to each target cluster sequence according to the matching degree between adjacent target clusters in each target cluster sequence, the similarity of the flow vectors between the adjacent target clusters, and the shape feature of the target cluster.

[0134] The needle tip position screening module 206 is configured to screen the puncture needle region sequence from all target cluster sequences based on all puncture needle possible indicators, and screen the needle tip position from each target cluster in the puncture needle region sequence;

[0135] The needle tip motion direction displacement evaluation module 207 is configured to evaluate the needle tip motion direction and the needle tip displacement of the puncture needle at the next moment according to the distance between the needle tip positions in adjacent puncture needle regions in the puncture needle region sequence and the optical flow vector corresponding to the pixel points in the puncture needle region.

[0136] Figure 3 is a structural schematic diagram of a computer device provided by an embodiment of the present application. As shown in the example, Figure 3 the computer device 300 includes a memory 301, a processor 302, and a computer program 303 stored in the memory 301 and running on the processor 302, wherein the processor 302 executes the computer program 303, so that the computer device can execute any one of the above-mentioned intervention puncture ultrasound assisted positioning methods.

[0137] Based on the same inventive concept as the above method embodiments, the present application provides a server including a memory and a processor. The memory is configured to store executable program code, and the processor is configured to call and run the executable program code from the memory, so that the device executes any one of the above-mentioned intervention puncture ultrasound assisted positioning methods.

[0138] Based on the same inventive concept as the above method embodiments, the present application provides a computer program product, which includes computer program code. When the computer program code runs on a computer, it makes the computer execute any one of the above-mentioned intervention puncture ultrasound assisted positioning methods.

[0139] Based on the same inventive concept as the above method embodiments, the present application provides a computer readable storage medium, which stores computer program code. When the computer program code runs on a computer, it makes the computer execute any one of the above-mentioned intervention puncture ultrasound assisted positioning methods.

[0140] In summary, compared with evaluating the movement of the puncture needle at the next moment by observing the ultrasound image collected at the current moment, when evaluating the movement of the puncture needle at the next moment, the present application quantifies the matching degree and the puncture needle possible indicators, thereby realizing the positioning of the needle tip position, and relatively objectively realizing the evaluation of the needle tip motion direction and the needle tip displacement of the puncture needle at the next moment, thereby improving the accuracy of evaluating the movement of the puncture needle at the next moment, and further improving the accuracy of intervention puncture ultrasound positioning tracking.

[0141] The above examples are only used to illustrate the technical solutions of the present application, and are not intended to limit the present application; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that the technical solutions recorded in the foregoing examples can be modified, or some technical features can be replaced by equivalent features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. An interventional puncture ultrasound-assisted positioning system, characterized in that, It includes a processor and memory. The processor processes instructions stored in memory to implement: During the interventional puncture process, ultrasound images are acquired in real time using an ultrasound probe. Based on the motion between each frame of ultrasound image and the next frame, the optical flow vector corresponding to each pixel in each frame of ultrasound image is determined. Based on the similarity between the optical flow vectors corresponding to pixels in each frame of ultrasound image and the distance between pixels, the pixels in each frame of ultrasound image are clustered to obtain target clusters; Based on the distribution of non-overlapping pixels between each target cluster and the target cluster in the next frame of the ultrasound image to which it belongs, as well as the center of each target cluster and the optical flow vector corresponding to the pixels within each target cluster, the matching degree between each target cluster and the target cluster in the next frame of the ultrasound image to which it belongs is determined. This includes: designating any target cluster as a first labeled cluster; designating any target cluster in the next frame of the ultrasound image to which the first labeled cluster belongs as a second labeled cluster; designating the region in the next frame of the ultrasound image to which the first labeled cluster belongs as a reference region; designating the center of the reference region as a reference center; and designating the intersection of the second labeled cluster and the reference region as a reference intersection. The region is defined as follows: all pixels in the second marker cluster except for the reference intersection region constitute the center of the region and are determined as candidate centers. Pixels in the second marker cluster except for the reference intersection region represent non-overlapping pixels between the first and second marker clusters. The direction from the reference center to the candidate center is determined as the reference motion direction. The mean of the optical flow vectors corresponding to all pixels in the first marker cluster is determined as the candidate motion vector, and the direction of the candidate motion vector is determined as the candidate motion direction. The matching degree between the first and second marker clusters is determined based on the angle between the reference motion direction and the candidate motion direction, the Euclidean distance between the reference center and the center of the second marker cluster, and the number of pixels in the reference intersection region. Based on the degree of matching between target clusters, mutually matching target clusters are selected from all ultrasound images to form a target cluster sequence; Based on the matching degree between adjacent target clusters in each target cluster sequence, the similarity of optical flow vectors between adjacent target clusters, and the shape features of the target clusters, the possible indicators of the puncture needle corresponding to each target cluster sequence are determined. This includes: determining the directional consistency of each target cluster sequence based on the matching degree between adjacent target clusters in each target cluster sequence and the difference in the direction of optical flow vectors between adjacent target clusters; determining the deviation distance of each pixel in each target cluster and the center of its respective target cluster as the Euclidean distance; determining the shape feature index of each target cluster based on the variance of the deviation distances of all pixels in each target cluster; and determining the possible indicators of the puncture needle corresponding to each target cluster sequence based on the directional consistency of each target cluster sequence and the shape feature index of all target clusters in each target cluster sequence. Here, both the directional consistency and the shape feature index are positively correlated with the possible indicators of the puncture needle. Based on all possible indicators of the puncture needle, the puncture needle region sequence is selected from all target cluster sequences, and the needle tip position is selected from each target cluster in the puncture needle region sequence. This includes: determining any target cluster in the puncture needle region sequence as a marker region, constructing a marker ray with the center of the marker region as the starting point and the direction of the mean of the optical flow vectors corresponding to all pixels in the marker region as the extension direction; selecting the pixel farthest from the center of the marker region from the intersection of the marker ray and the marker region as the needle tip pixel, and determining the position of the needle tip pixel as the needle tip position. Based on the distance between the needle tip positions in adjacent puncture needle regions in the puncture needle region sequence, and the optical flow vector corresponding to the pixel in the puncture needle region, the needle tip movement direction and needle tip displacement at the next moment are evaluated.

2. The interventional puncture ultrasound-assisted positioning system according to claim 1, characterized in that, Based on the similarity between the optical flow vectors corresponding to pixels in each frame of ultrasound image, and the distance between pixels, the pixels in each frame of ultrasound image are clustered to obtain target clusters, including: The normalized value of the cosine similarity between the optical flow vectors corresponding to every two pixels in each frame of ultrasound image is determined as the vector similarity factor between every two pixels in each frame of ultrasound image. The absolute value of the difference between the magnitudes of the optical flow vectors corresponding to every two pixels in each frame of ultrasound image is determined as the motion distance factor between every two pixels in each frame of ultrasound image. The consistency coefficient between every two pixels in each frame of ultrasound image is determined based on the vector similarity factor, motion distance factor, and Euclidean distance between every two pixels in each frame of ultrasound image. Among them, the vector similarity factor is positively correlated with the consistency coefficient, while the motion distance factor and Euclidean distance are both negatively correlated with the consistency coefficient. Based on the consistency coefficient among all pixels in each frame of ultrasound image, the pixels in each frame of ultrasound image are clustered, and each resulting cluster is denoted as the target cluster.

3. The interventional puncture ultrasound-assisted positioning system according to claim 1, characterized in that, The formula corresponding to the degree of matching between the first and second marker clusters is: Where D represents the degree of matching between the first and second marker clusters; It is the normalization function; n is the number of pixels in the reference intersection region; It is the natural exponential function; is the angle between the reference motion direction and the candidate motion direction; d is the Euclidean distance between the reference center and the center of the second marker cluster.

4. The interventional puncture ultrasound-assisted positioning system according to claim 1, characterized in that, The formula for directional consistency corresponding to the target cluster sequence is: ;in, It represents the directional consistency corresponding to the r-th target cluster sequence; r is the sequence number of the target cluster sequence. It is a normalization function; is the number of target clusters in the r-th target cluster sequence; t is the index of the target cluster in the r-th target cluster sequence; It represents the degree of matching between the t-th target cluster and the (t+1)-th target cluster in the r-th target cluster sequence; It is the cosine similarity between the motion representative vectors of the t-th target cluster and the (t+1)-th target cluster in the r-th target cluster sequence; the motion representative vectors of the target clusters are the mean of the optical flow vectors corresponding to all pixels in the target cluster.

5. The interventional puncture ultrasound-assisted positioning system according to claim 1, characterized in that, Based on the directional consistency corresponding to each target cluster sequence and the shape feature indices corresponding to all target clusters within each target cluster sequence, the possible indices of the puncture needle corresponding to each target cluster sequence are determined, including: The sum of the shape feature indices corresponding to all target clusters within each target cluster sequence is used to determine the possible puncture needle shape factors for each target cluster sequence. Normalize the product of the directional consistency and the possible factors of the puncture needle shape corresponding to each target cluster sequence to obtain the possible index of the puncture needle corresponding to each target cluster sequence.

6. The interventional puncture ultrasound-assisted positioning system according to claim 1, characterized in that, The formulas for the direction of needle tip movement and the needle tip displacement at the next moment are as follows: ； Where R is the direction of the predicted needle tip movement in the next moment; L is the predicted needle tip displacement in the next moment; M is the number of puncture needle regions in the puncture needle region sequence; m is the sequence number of the puncture needle regions in the puncture needle region sequence; Q is the cumulative value of the sequence numbers of all puncture needle regions in the puncture needle region sequence. It is the mean of the optical flow vectors corresponding to all pixels in the m-th puncture needle region sequence; It is the distance between the needle tip position in the m-th puncture needle region and the needle tip position in the (m+1)-th puncture needle region sequence.

Images