needle position

Abstract

A method includes receiving data corresponding to a plurality of radiographic imaging slices of a body; determining a location of a needle insertion in the body based on combining information from at least one radiographic imaging slice that includes an indication of a first portion of the needle outside the body and at least another radiographic imaging slice that includes an indication of a second portion of the needle inside the body, wherein the combined needle region is generated by merging data corresponding to a location of the first portion of the needle outside the body and data corresponding to a location of the second portion of the needle inside the body; generating display data for providing a visual representation of the needle in an image of the body in combination with a visual representation of at least the first portion and the second portion of the needle superimposed on the image, wherein the image is in a plane that is digitally tilted with respect to a plane parallel to the plurality of radiographic imaging slices.

Description

TECHNICAL FIELD

[0001] The present invention relates to methods, apparatuses, and tangible machine-readable media for determining needle position, for example, during a computed tomography (CT) guided interventional procedure. BACKGROUND

[0002] A body of a subject can be imaged, for example, using a CT imaging apparatus, as part of an interventional procedure such as a CT guided needle biopsy, in which a needle is inserted into the body during imaging. A user, such as a physician or radiographer, can need to identify the position of the needle in the body of the subject. Radiographic imaging slices obtained by the CT imaging apparatus can provide an indication of the position of the needle in the body of the subject. For example, the radiographic imaging slices can provide information about the direction of the needle, the position of the needle tip, and / or the entry point of the needle on the skin surface of the subject. This information can help provide guidance to the user to determine the needle angle (or needle trajectory) that deviates from the planned needle path, the distance of the needle tip from a target such as a lesion in the body, and / or the length of the needle that has been inserted.

[0003] CT guided needle biopsy of a lesion utilizes CT radiographic imaging slices that provide detailed information for the user to make certain decisions about the biopsy. For example, the CT radiographic imaging slices can provide high density resolution and spatial resolution that enables accurate positioning of the lesion and facilitates clear understanding of, for example, the condition of the soft tissue inside and around the lesion, thereby avoiding certain structures or necrotic tissue.

[0004] In certain interventional procedures, the user can obtain two-dimensional radiographic imaging slices taken along an axial imaging plane to observe the biopsy target on a user interface. The user can rely on their experience to plan an interventional path based on analysis of multiple radiographic imaging slices taken along the axial plane and / or the user can perform the interventional procedure with physical tilting in which the imaging apparatus is tilted with respect to the body of the subject to obtain radiographic imaging slices along a particular plane that can be different from the axial plane.

[0005] Planning and conducting an interventional procedure can be a relatively time consuming task because complete information about the position of the needle can not be identified from a single radiographic imaging slice. For example, one radiographic imaging slice can contain information about the needle tip while another radiographic imaging slice can contain information about another portion of the needle. Thus, the actual interventional path can span multiple radiographic imaging slices. Therefore, the complete interventional path can not be displayed in a single image on a user interface. Thus, the user can rely on their experience and / or obtain additional radiographic imaging slices to ensure that the user can accurately identify the position of the needle. However, additional scans require time and can increase the radiation dose.

[0006] A user viewing radiographic imaging slices one at a time can find it challenging and / or time consuming to find information needed to safely and / or reliably run an interventional procedure because multiple radiographic imaging slices can need to be evaluated before deciding how to proceed with the interventional procedure. SUMMARY

[0007] Various aspects or embodiments described herein relate to improving determination and / or visualization of a location of a needle in a body. Various aspects or embodiments described herein can avoid one or more problems associated with using radiographic imaging slices to plan and run an interventional procedure.

[0008] In a first aspect, a method is described. The method is a computer- implemented method. The method includes receiving data corresponding to a plurality of radiographic imaging slices of a body. The method also includes determining a location of a needle insertion in the body. The determination can be based on combining information from at least one radiographic imaging slice of the radiographic imaging slices that includes an indication of a first portion of the needle outside the body and at least another radiographic imaging slice of the radiographic imaging slices that includes an indication of a second portion of the needle inside the body. A combined needle region is generated by merging data corresponding to a location of the first portion of the needle outside the body and data corresponding to a location of the second portion of the needle inside the body. The method also includes generating display data for providing a visual representation of the needle in an image of the body in combination with a visual representation of at least the first portion and the second portion of the needle superimposed on the image. The image is in a plane that is digitally tilted with respect to a plane parallel to the plurality of radiographic imaging slices.

[0009] In some embodiments, the method includes causing a user interface to display the image.

[0010] In some embodiments, determining the location of the needle includes fitting a line to a plurality of regions in the received data indicating the locations of the first portion and the second portion of the needle.

[0011] In some embodiments, determining the location of the needle includes determining a true needle region from a plurality of candidate needle regions, which can be determined by minimizing an energy function derived from the received data.

[0012] In some embodiments, the energy function is based on at least one of: a degree of dispersion of the candidate needle region; an area of the candidate needle region; an average pixel value of an edge of the candidate needle region; an average pixel value of an interior portion of the candidate needle region; an average of all cross-sectional values of the candidate needle region; a circularity parameter; and a skewness parameter.

[0013] In some embodiments, the method includes identifying the first portion of the needle outside the body by performing a morphological opening operation on received data corresponding to the first portion of the needle. The method can also include determining a three-dimensional region corresponding to the first portion of the needle based on a planned path for the needle. The three-dimensional region can include a plurality of candidate needle regions from which the true needle region can be determined.

[0014] In some embodiments, the method includes identifying the second portion of the needle inside the body by performing a threshold cut on received data corresponding to the second portion of the needle. The method can include determining a three-dimensional region corresponding to the second portion of the needle based on a planned path for the needle. The three-dimensional region can include a plurality of candidate needle regions from which the true needle region can be determined.

[0015] In some embodiments, the method includes expanding the three-dimensional region corresponding to the second portion of the needle. Expanding the three-dimensional region can include searching for at least one neighboring candidate needle region removed by the threshold cut. Expanding the three-dimensional region can also include data corresponding to the neighboring candidate needle region as part of the expanded three-dimensional region.

[0016] In some embodiments, the method includes detecting a tip of the needle, which can be based on a comparison of a measurement within a candidate needle tip region within the data to a threshold value indicative of an absence of needle structure within the candidate needle tip region.

[0017] In some embodiments, if the comparison to the threshold value indicates a presence of needle structure in the candidate needle tip region, another candidate needle tip region is identified from the data to determine whether the other candidate needle tip region includes data indicative of a presence of needle structure. If the comparison to the threshold value indicates an absence of needle structure within the data corresponding to the candidate needle tip region, a previously identified candidate needle tip region including data indicative of a presence of needle structure within the previously identified candidate needle tip region can be determined to contain the tip of the needle.

[0018] In some embodiments, the method comprises detecting a point of insertion of the needle on the body. Detecting the point of insertion can be based on a first line fitted to a predicted trajectory of the needle. The predicted trajectory of the needle can be determined based on the determined position of the needle and a second line fitted along a surface of the body.

[0019] In some embodiments, the method comprises causing a user interface to provide an indication of a difference between a predicted trajectory of the needle and a planned trajectory. The indication of the difference can be provided in response to determining that there is a deviation between the predicted trajectory and the planned trajectory.

[0020] In a second aspect, a method is described. The method is a computer- implemented method. The method comprises receiving data corresponding to at least one radiographic imaging slice of a body, the data comprising information indicative of a position of a needle inserted in the body. The method further comprises determining a predicted trajectory of the needle based on the information. The method further comprises generating display data for providing a visual representation of the needle in an image plane parallel to a line comprising the predicted trajectory of the needle, wherein the image plane is tilted with respect to the at least one radiographic imaging slice of the body.

[0021] In a third aspect, an apparatus is described, the apparatus being for visualizing a needle inserted in a body. The apparatus comprises a processing unit configured to implement the method of any of the first aspect and / or the second aspect and / or embodiments thereof. The apparatus can determine a position of the needle. The apparatus further comprises a display unit configured to display an image of the body overlaid with the position of the needle.

[0022] In a fourth aspect, a tangible machine-readable medium is described. The tangible machine-readable medium comprises instructions which, when run on at least one processor, cause the at least one processor to implement the method of any of the first aspect and / or the second aspect and / or embodiments thereof.

[0023] These and other aspects of the application will be apparent from and elucidated with reference to the embodiments described hereinafter. BRIEF DESCRIPTION OF DRAWINGS

[0024] Exemplary embodiments of the present application will now be described, by way of example only, with reference to the following drawings:

[0025] Figure 1 relating to a method of determining a position of a needle according to an embodiment;

[0026] Figure 2 relating to a method of determining a position of a needle according to an embodiment;

[0027] Figure 3 This relates to a method for determining the position of a needle according to an embodiment;

[0028] Figure 4 This is a schematic diagram of a system for determining needle position according to an embodiment;

[0029] Figures 5a to 5b These are images used in certain methods described in the embodiments;

[0030] Figures 6a to 6b It is an image used to visualize the position of the needle according to an embodiment;

[0031] Figures 7a to 7b It is an image used to visualize the position of the needle according to an embodiment;

[0032] Figure 8a and Figure 8b Images for visualizing the position of a needle and corresponding illustrations for determining the position of a needle, according to embodiments, are shown respectively;

[0033] Figure 9 This relates to a method for determining the position of a needle according to an embodiment;

[0034] Figure 10 This relates to a method for determining the position of a needle according to an embodiment;

[0035] Figure 11 An image for visualizing the position of the needle is shown according to an embodiment;

[0036] Figure 12 This is a schematic diagram of a device for determining the position of a needle according to an embodiment; and

[0037] Figure 13 This is a schematic diagram of a machine-readable medium for determining the position of a needle according to an embodiment. Detailed Implementation

[0038] Figure 1 A method 100 for determining the position of a needle within the body of an object is illustrated (e.g., a computer-implemented method). Method 100 may be implemented by a computer (e.g., a user computer communicatively coupled to a user interface) or a server or cloud-based service (e.g., communicatively coupled to a user computer and / or a user interface).

[0039] The method 100 includes receiving data corresponding to a plurality of radiographic imaging slices of a body at block 102. The plurality of radiographic imaging slices can be obtained by causing a CT imaging device to image a plurality of parallel cross-sections of the body. The CT imaging device can be communicatively coupled to a computer implementing the method 100 to cause the CT imaging device to perform the imaging and / or to receive data from the CT imaging device. The computer implementing the method 100 can be separate or the same as a computer used to construct the radiographic imaging slices for visualization through a user interface.

[0040] The plurality of radiographic imaging slices can be parallel to each other. The radiographic imaging slices can be obtained with respect to a certain plane of the body. For example, the radiographic imaging slices can be taken in an axial plane perpendicular to a long axis defined by a length of the body. Via multiplanar reformation (MPR), the data obtained with respect to the radiographic imaging slices corresponding to the axial plane (or another plane) can be transformed to enable visualization of the body in a different plane (e.g., a sagittal plane, a coronal plane, or another plane). In either case, when generating data for reconstructing a visual representation of the body imaged by the imaging device, the data from the plurality of radiographic imaging slices can be combined or merged to create a three-dimensional representation of the imaged portion of the body that can be displayed on a user interface.

[0041] The method 100 also includes determining a location of a needle inserted in the body based on combining information from at least one of the radiographic imaging slices including an indication of a first portion of the needle outside the body and at least another of the radiographic imaging slices including an indication of a second portion of the needle inside the body at block 104. Determining the location of the needle includes generating a combined needle region by merging data corresponding to a location of the first portion of the needle outside the body and data corresponding to a location of the second portion of the needle inside the body.

[0042] The method 100 also includes generating display data for providing a visual representation of the needle in an image of the body in combination with visual representations of at least the first portion and the second portion of the needle superimposed on the image at block 106. The image is in a plane that is digitally tilted with respect to a plane parallel to the plurality of radiographic imaging slices.

[0043] When performing an interventional procedure, a needle can be inserted into a body in a trajectory (i.e., a direction) specified by a user. The trajectory can be determined according to the user’s experience depending on the clinical scenario, e.g., to avoid certain tissues in the body. In case the radiographic imaging slices are taken in an axial plane and the needle trajectory is tilted with respect to the axial plane, different portions of the needle can be detected in each of the radiographic imaging slices. In other words, a single radiographic slice can not contain all information about the needle’s position.

[0044] By combining information about a first portion of the needle outside the body and information about a second portion of the needle inside the body from the radiographic imaging slices, the needle’s position can be determined. The needle’s position can provide information about the length of the needle inserted into the body, the distance of the needle tip (or “point”) from a target such as a lesion in the body, the needle’s trajectory (or direction) with respect to certain features of the body (e.g., a certain tissue), the entry point of the needle on the surface of the body.

[0045] Since the information relates to both the first portion and the second portion from which the needle’s trajectory can be determined (e.g., based on the relative difference of the positions of the first portion and the second portion), the needle’s position can be determined more easily or more accurately compared to the case of viewing a single radiographic imaging slice at a time (e.g., on a user interface).

[0046] According to the method 100, a digital tilt procedure can be used to generate the images. The digital tilt procedure is described in more detail below with respect to Figure 9 The digital tilt procedure is described in more detail below with respect to

[0047] Figure 2 A method 200 (e.g., a computer-implemented method) of determining a position of a needle in a body of a subject (which can be used to facilitate visualization of the needle in an image) is shown. The method 200 can be implemented by a computer (e.g., a user computer communicatively coupled to a user interface) or a server or cloud-based service (e.g., communicatively coupled to a user computer and / or a user interface). In some embodiments, the method 200 can be combined with the method 100 of Figure 1 Thus, for ease of reference, blocks 102-106 of the method 100 are shown in Figure 2 Figure 1

[0048] ​​In some embodiments, the method 200 includes causing a user interface to display an image of the body (e.g., from display data generated at block 106 of the method 100) in combination with a visual representation of at least the first portion and the second portion of the needle superimposed on the image (e.g., a length of the needle sufficient to enable the user to determine the trajectory of the needle can be displayed (e.g., the entire length of the needle)). In some embodiments, the image displayed on the user interface can include an MPR image and / or a three-dimensional reconstruction of the body, which can be arbitrarily rotated according to user demand. When viewing the image, the user can decide how to proceed with the interventional procedure. Since more information about the needle can be apparent from the image, the user can more accurately, quickly, and / or efficiently implement the interventional procedure compared to a situation where the image is viewed that contains partial information about the needle (e.g., as can be the case when individual radiographic imaging slices are viewed). As the interventional procedure continues, the user interface can be automatically updated in response to further data received from the imaging device.

[0049] In some embodiments, determining the position of the needle includes merging data corresponding to the position of the first portion of the needle outside the body and data corresponding to the position of the second portion of the needle inside the body. The merged data can be used to generate a combined needle region from the received data. The method 100 can further include using the combined needle region to determine the position of the needle.

[0050] In some embodiments, determining the position of the needle includes fitting a line to a plurality of regions in the data received indicating the position of the first portion and the second portion of the needle. If the (relative) positions of the first portion and the second portion can be ascertained, the trajectory of the needle can be determined. By fitting a line to portions of the data indicating the position of the first portion and the second portion, the trajectory of the needle can be determined.

[0051] Figure 3 A method 300 (e.g., a computer-implemented method) of determining a position of a needle in a body of a subject is shown. The method 300 can be implemented by a computer (e.g., a user computer communicatively coupled to a user interface) or a server or cloud-based service (e.g., communicatively coupled to a user computer and / or a user interface). In some embodiments, the method 300 includes the method 100 and / or the method 200.

[0052] The method 300 is initialized at block 302. Other blocks of the method 300 are described below.

[0053] At block 304 of the method 300, the user can input a plan for the interventional procedure. For example, prior imaging by the imaging device and / or physical examination of the body can provide information to the user to enable the user to formulate a plan for how to perform the interventional procedure. The plan can include a target point (e.g., a lesion in the body) and an entry point on the surface of the body. Assuming the entry point is accurately aimed, certain methods described herein can provide the user with sufficient information to be able to determine a trajectory for the needle insertion into the body so that the user can determine whether the trajectory needs to be changed to reach the target point according to the plan. Additionally, certain methods described herein can provide information about the length of the needle insertion into the body and / or the location of the needle tip so that the user can determine how far the needle tip is from the target point.

[0054] At block 306 of the method 300, a first portion of the needle is detected. At block 308 of the method, a second portion of the needle is detected. Blocks 306 and 308 can be performed in any order and can be implemented based on data received from the imaging device (e.g., at block 102 of the method 100).

[0055] In some embodiments, block 306 of the method 300 includes identifying the first portion of the needle outside the body by performing a morphological opening operation on the received data corresponding to the first portion of the needle (or any other appropriate image processing algorithm can be performed) to extract the needle region outside the body and any other region that looks like a needle, for example. The method 300 further includes determining a three-dimensional region corresponding to the first portion of the needle based on the planned path of the needle. The three-dimensional region can include a plurality of candidate needle regions from which the true needle region can be determined. For example, a region of interest can be based on the planned path and the region of interest can be extended to a volume of interest that includes a plurality of candidate needle regions (each candidate needle region can be labeled as “i” in the description below).

[0056] A candidate needle region can include an area (e.g., an area) of pixels in the data (from each radiographic imaging slice). A candidate needle region can or can not have a fixed size or shape. However, certain conditions on the size or shape can depend on the configuration of the CT imaging device. As described below, among the plurality of candidate regions, the region that most looks like a needle can be determined as the “true needle region” via an energy calculation.

[0057] In some embodiments, the candidate needle regions can be filtered based on pixel values in the data, as there can be a large difference in pixel values between data points corresponding to the needle (which can be a metal needle) and other data points corresponding to other components outside the body. For example, the filtering can include comparing the detected pixel values to a threshold pixel value and ignoring any candidate needle region that does not appear to correspond to the presence of a needle structure.

[0058] In some embodiments, block 308 of method 300 includes identifying the second portion of the needle inside the body by performing thresholding on the received data corresponding to the second portion of the needle (or any other appropriate image processing algorithm can be performed). Method 300 also includes determining a three-dimensional region corresponding to the second portion of the needle based on the planned path of the needle. As described in block 306, the three-dimensional region can include a plurality of candidate needle regions from which the true needle region can be determined.

[0059] In some embodiments, method 300 includes extending the three-dimensional region corresponding to the second portion of the needle at block 310 by: searching for at least one adjacent candidate needle region removed by the thresholding, and including data corresponding to the adjacent candidate needle region as part of the extended three-dimensional region.

[0060] In some embodiments, determining the position of the needle includes determining the true needle region from the plurality of candidate needle regions by minimizing an energy function derived from the received data.

[0061] In some embodiments, the energy function is based on at least one of: a degree of dispersion of the candidate needle region; an area of the candidate needle region; an average pixel value of an edge of the candidate needle region; an average pixel value of an inner portion of the candidate needle region; an average of all cross-sectional values of the candidate needle region; a circularity parameter; and a deviation parameter.

[0062] In some embodiments, the circularity parameter can be based on whether the candidate needle region is circular. In other words, the circularity parameter can provide a quantitative indication of whether the candidate needle region is circular.

[0063] In some embodiments, the deviation parameter can be based on whether there is a deviation between the planned trajectory of the needle and the measured trajectory of the needle.

[0064] A description of the calculation of the energy function and how it is used to determine the needle region is now given. This calculation can be performed in the two blocks 306, 308 described above.

[0065] In the case of block 306 (for the portion of the needle outside the body), in some embodiments, the energy function can be represented as:

[0066] where

[0067] and

[0068] and where

[0069] α, β, γ, δ, ε, and θ are the weights of each feature. The weights of each feature can be fixed values ​​determined experimentally. e(i) is the dispersion of the candidate needle region i. Dispersion refers to how the points in the region are distributed or clustered into lines. If the points are uniformly distributed in space, the value of e(i) is low, while if the points form lines, the value of e(i) is high. a(i), d(i), c(i), and n(i) refer to the cross-section (j∈{1,2,……N)) in the direction perpendicular to i. i The average value of all data points calculated on}). Here, i refers to the 3D candidate needle region comprising multiple cross-sections, and j refers to the cross-section of that region. a(i) is the average circular area (of the candidate needle region), and d(i) is the average pixel value of the edges (of the candidate needle region). n(i) is the average pixel value of the inner region (i.e., pixels not at the edges of the candidate needle region). r(i) describes whether the area is circular (i.e., the "roundness parameter"). For region "i", c(i) is the average value of all cross-sections j. For example, see the formula above, where P j S is the perimeter of the cross-sectional region j of a candidate needle region i. i and L i These are the minor and major axes of the region projected along the regional direction, which refers to the needle direction of the actual needle region (for other candidate needle regions, the regional direction refers to the longest direction of the region). v(i) is the deviation between the planned direction and the actual direction (i.e., the "deviation parameter"). These are the standard values ​​for each feature. These "standard values" refer to fixed values ​​(e.g., derived from experience). For example, a standard value could refer to the value that the real needle region is considered most likely to possess. By subtracting the "standard values" from the calculated values, the real needle region will have the minimum value of E(i).

[0070] Once the energy function is determined, an optimization problem is performed to obtain the true needle region from multiple candidate needle regions, i.e.,

[0071] argminE(i)

[0072] The process for obtaining the true needle region of the portion of the needle inside the body is similar to that for obtaining the true needle region of the portion of the needle outside the body. However, when extracting candidate needle regions, threshold truncation replaces morphological opening operations, and the energy function is different due to the difference between data for the needle inside the body and data for the needle outside the body. In some embodiments, the energy function is expressed as:

[0073]

[0074] where the terms in the expression are as described above. In block 310, a neighborhood search can be used to expand the needle region in view of previous thresholding processes that can otherwise result in omission of certain regions of interest. Again, an optimization problem is executed to obtain the true needle region inside the body.

[0075] At block 312 of the method, the needle region inside the body can be combined with the needle region outside the body (e.g., as described in block 104) to provide an accurate determination of the needle position (e.g., including trajectory or direction). An accuracy result can be selected to represent the needle direction. For example, a comparison between the true needle direction and the planned needle direction can be used to determine the accuracy of the needle positioning.

[0076] As part of the flow for determining the position of the needle and thus the trajectory or direction of the needle, at block 314, a line fitting can be performed on the true needle region. The coordinates of the detected region i can be represented as:

[0077] {X,Y,Z} e {(x i1 ,y i1 ,z i1 ),……(x ij ,y ij ,z ij )}

[0078] This means that the number of points in the candidate needle region i is j. By fitting a line using {X,Y,Z}, a point on the needle (x0,y0,z0) and a unit direction vector of the needle (v x ,v y ,v z ) can be obtained. This information can be used to construct a visual representation of the needle position for an image to be displayed on a user interface.

[0079] In some embodiments, the method 300 includes detecting a tip of the needle at block 316 based on a comparison of measurements within a candidate needle tip region within the data to a threshold value indicative of an absence of needle structure within the candidate needle tip region.

[0080] In some embodiments, if the comparison to the threshold value indicates a presence of needle structure in the candidate needle tip region, another candidate needle tip region is identified from the data to determine whether the other candidate needle tip region includes data indicative of a presence of needle structure. If the comparison to the threshold value indicates an absence of needle structure within the data corresponding to the candidate needle tip region, the previously identified candidate needle tip region including data indicative of a presence of needle structure within the previously identified candidate needle tip region is determined to contain the tip of the needle.

[0081] A further description of the needle tip detection flow is given below.

[0082] In some embodiments, the method 300 comprises detecting, at block 318, a point of insertion (e.g., an "entry point") of the needle on the body based on the first line fitted to the predicted trajectory of the needle, the predicted trajectory of the needle being determined based on the determined position of the needle and the second line fitted along the surface of the body.

[0083] Further description of the insertion point detection procedure is given below.

[0084] In some embodiments, the method 300 comprises causing, at block 320, the user interface to provide an indication of a difference between the predicted trajectory and the planned trajectory in response to determining that there is a deviation between the predicted trajectory and the planned trajectory. For example, the user can identify the deviation from the indication and, if appropriate, make changes to the intervention procedure to ensure that the target point is reached when further inserting the needle.

[0085] At block 322, the method 300 can end or at least some blocks of the method 300 are repeated (e.g., when further data is received from the imaging device).

[0086] Figure 4 A system 400 for implementing certain methods described herein is depicted. The system 400 is used to obtain radiographic imaging slices involved in the method and to perform certain blocks of the methods described herein.

[0087] In the system 400, an intervention procedure is depicted as being performed on a subject 402, in particular on a certain part 404 of the subject's body (indicated by the dashed line in Figure 4 The subject is supported by a support 406, such as a couch. The position of the support 406 can be controlled (e.g., by a computer) in relation to an imaging device 408 (e.g., a CT imaging device) of the system 400. The imaging device 408 is used to obtain radiographic imaging slices. For example, the imaging device 408 can comprise an emitter (e.g., an X-ray emitter, not shown) and a corresponding detector (not shown) mounted on a gantry (which can form part of the imaging device 408) that is rotatable to enable a series of measurements to be made to obtain data for constructing each radiographic imaging slice. The imaging device 408 and the support 406 can be moved relative to each other to obtain a plurality of radiographic imaging slices.

[0088] The intervention procedure of inserting a needle 410 (e.g., supported by a needle carriage 412) into the body of the subject 402 can be performed at the same time as the radiographic imaging slices are obtained.

[0089] The system 400 also includes a computer 414 (e.g., a processing unit including processing circuitry) for implementing certain methods described herein. The computer 414 is communicatively coupled to a user interface 416, e.g., a display for visualizing images obtained by the imaging device 408 and / or displaying information generated or determined by the computer 414.

[0090] While the system 400 can appear to depict the imaging device 408, the computer 414, and the user interface 416 as being located at the same location, in some cases these components can be located at the same or different locations. For example, the imaging device 408 can be located at a different location than the computer 414 and / or the user interface 416. The computer 414 can be implemented by a user computer (e.g., connected to the same terminal as the user interface 416), or can be implemented by a server or cloud-based service.

[0091] Some experimental images obtained according to certain methods or systems described herein are described below. For ease of reference, certain features corresponding to features depicted in the system 400 are described in Figures 5a to 5b with reference numerals incremented by 100, certain features corresponding to features depicted in the system 400 are described in Figures 6a to 6b with reference numerals incremented by 200, certain features corresponding to features depicted in the system 400 are described in Figures 7a to 7b with reference numerals incremented by 300.

[0092] Figures 5a to 5b Different (CT) radiographic imaging slices obtained by an imaging device (e.g., an imaging device as shown in Figure 4 is shown. Figures 5a to 5b An insertion process of a needle 510 into a body of a subject 502 is shown. In Figure 5a , a “first” portion of the needle 510 is visible outside the body. In Figure 5b , a “second” portion of the needle 510 is visible inside the body. In this case, Figure 5a refers to a seventh radiographic imaging slice, and Figure 5b refers to a tenth radiographic imaging slice of a plurality of radiographic imaging slices. Thus, when a user attempts to manually determine the position of the needle 510, the user can need to cycle between these different slices.

[0093] Figures 6a to 6b Further images are shown in which certain methods described herein have been implemented to enable visualization of the entire needle 610 within a single image. In Figures 6a to 6b , an entry point 630 on the surface of the body 602 and a needle tip location 632 can be identified to assist a user in performing an intervention procedure. In Figures 6a to 6bThe line 634 is also shown indicating the trajectory of the needle 610.

[0094] Experimental data was obtained when performing interventional procedures on 31 patients (122 series) using certain methods described herein. The data was collected using different instruments (e.g., imaging devices) from different countries, and included different body parts, e.g., chest, waist, abdomen, shoulder, etc. The accuracy of needle position determination according to certain methods described herein was determined from this data. The accuracy of needle direction detection was found to be 100%, the accuracy of needle tip detection was 91%, and the accuracy of entry point detection was 96%. Thus, certain methods described herein can provide accurate determination of needle position (e.g., including needle trajectory / direction, tip location, and entry point), which can facilitate efficient and accurate performance of interventional procedures by a user.

[0095] Figures 7a to 7b An implementation of needle tip detection in relation to radiographic imaging slices is depicted. The needle tip detection procedure can be based on the segmentation 740 of the needle 710 (e.g., each slice can contain a segmented portion of the needle 710). Thus, each candidate needle region can refer to one segmented portion of the needle 710. Since the segmentation 740 can not necessarily identify the true needle tip location, an additional procedure as described in block 316 can be used to identify the true needle tip 742.

[0096] In some embodiments, if there is any needle structure in a candidate needle tip region, a (virtual) cylinder can be created to detect further needle structure. The cylinder is aligned along the best fit line 744 of the segmented needle portion, and includes the candidate needle tip region. From the current end of the needle segmentation and within the cylinder, it is determined whether there is still needle structure beyond the segmentation. In other words, if a candidate needle tip region contains needle structure, but the adjacent candidate needle tip region (forward from the previously analyzed needle tip region) does not contain needle structure, the previously analyzed needle tip region can be considered to contain the tip of the needle, thereby identifying the location of the tip of the needle 742.

[0097] A further description of needle tip identification is now given.

[0098] When a candidate needle tip region is identified as containing needle structure, the method tracks along the direction of the needle (line 744). On each cross-section of the cylinder, a difference in pixel values between the center region 746 (i.e., the candidate needle tip region) and its surrounding region 748 is computed. If the difference is large enough (meaning that the average CT pixel value of the center region 746 is much higher than the average CT pixel value of the surrounding region), it is determined that there is still needle structure in that region. This difference is computed from the center region 746 to the surrounding region 748.The difference is clearly visible in the inset figure in Figure 7a Figure 7aThe embedded graph in FIG. 7B shows the contrast between the central region 746 and the surrounding region 748 in more detail. Thus, if needle structure is still present in this region, the tracking can continue. When needle structure cannot be detected (due to little or no difference between the central region 746 and the surrounding region 748), it is determined that the true needle tip 742 has been reached (since there is no needle structure to indicate that the needle tip has been reached in the previous region).

[0099] Figures 8a to 8b An embodiment of needle entry point detection in relation to a radiographic imaging slice is depicted. In this embodiment, the needle entry point detection involves fitting two lines to certain structures identified in the radiographic imaging slice, and identifying the intersection of the two lines. An embodiment of the needle entry point detection of this embodiment is described below.

[0100] The volume data is reconstructed in a direction perpendicular to the needle 810. A sagittal plane of the reconstructed volume data (as shown in FIG. 8A, but other planes such as axial or coronal planes can also be used) includes the entire needle based on the three-dimensional coordinates of the needle 810. Thus, a line (e.g., a linear line) can be fitted to the reconstructed volume data. Figure 8a

[0101] The surface of the body can be detected based on the needle direction in the image. According to a quadratic curve equation, a line 850 (e.g., a curve as shown in FIG. 8B) is fitted along the surface of the body of the subject: Figure 8b

[0102] y = ax + bx + c 2

[0103] where it is assumed that the surface of the body in the sagittal plane can be represented by such an expression. In other cases, different expressions such as a linear line equation can be used depending on the shape of the surface of the body.

[0104] As mentioned above, the needle in the sagittal plane can be considered as a linear (two-dimensional) line 852, and according to a linear line equation, the straight line equation is known based on the needle region:

[0105] y = kx + d

[0106] By combining the above expressions, the intersection corresponding to the needle entry point 854 can be determined. In some cases, due to the surface of the body can not be smooth and / or can not correspond to a quadratic equation, further image processing analysis can be needed to fine tune the entry point.

[0107] The above methods and systems relate to methods of determining needle position. Another method of determining needle position is described below and can be implemented by the system 400 of Figure 4

[0108] ​​​​Figure 9 A method 900 (e.g., a computer-implemented method) of determining a position of a needle in a body of a subject and / or facilitating visualization of the position of the needle is shown. The method 900 can be implemented by a computer (e.g., a user computer communicatively coupled to a user interface) or a server or cloud-based service (e.g., communicatively coupled to a user computer and / or a user interface). The method 900 can be implemented as part of or in conjunction with any other method described herein.

[0109] The method 900 comprises receiving, at block 902, data corresponding to at least one radiographic imaging slice of the body, the data comprising information indicative of a position of a needle inserted in the body. The data received at block 902 can correspond to the data received at block 102 of the method 100.

[0110] The method 900 comprises determining, at block 904, a predicted trajectory of the needle based on the information. The predicted trajectory can be determined using certain methods described herein (e.g., using the method 100).

[0111] The method 900 comprises generating, at block 906, display data for providing a visual representation of the needle in an image plane parallel to a line strip comprising a predicted trajectory of the needle. The image plane can be tilted with respect to the at least one radiographic imaging slice of the body. Thus, a user can select an image view at any angle (e.g., based on an MPR image or a three-dimensional reconstruction of the body) and the needle can be displayed in full in the image. Since certain methods described herein (e.g., the method 100) can enable determination of the position of the needle (and thus the direction, tip position, and entry point of the needle), the visual representation of the needle can be included within any image position selected by the user. In case the needle is at an angle to the plane of the radiographic imaging slice (e.g., in an axial plane), the image can be digitally tilted in order to visualize the entire needle in the body. For example, the image (corresponding to the display data) lies in a plane that is digitally tilted with respect to a plane parallel to the at least one radiographic imaging slice. In other words, the generated image display data enables visualization of at least a portion of the needle in a digitally reconstructed plane at an angle to the plane of the radiographic imaging slice. The tilt / angle can vary according to user needs. This can avoid the need to perform a physical tilt (in which a gantry of an imaging device is tilted and aligned with the needle to image the entire needle), thereby saving time.

[0112] A digital tilt procedure can be used instead of a physical tilt procedure. Digital tilt can facilitate needle visualization in a specific or preferred direction / plane, which can provide flexible, intuitive, and / or convenient needle visualization. Additionally, a digital tilt procedure can be used to determine the exact (or exact) angle and depth of the needle, and can represent the actual needle path during the intervention procedure in its entirety. The path can be adjusted at any time, and the scan range can be precisely positioned to control the scan dose delivered by the imaging device, enabling accurate sampling to create the basis for pathological diagnosis of the lesion and also reducing unnecessary radiation damage to the subject.

[0113] A digital tilt procedure can involve planning the intervention procedure (i.e., localization of the lesion and design of the puncture path) and confirmation. By automatically identifying the actual puncture path and presenting a path image, the current needle position, direction, and needle angle can be confirmed. Such information can be useful for determining the distance of the needle tip from the lesion localization.

[0114] Figure 10 A method 1000 (e.g., a computer-implemented method) of determining a position of a needle in a body of a subject is shown. The method 1000 can be implemented by a computer (e.g., a user computer communicatively coupled to a user interface) or a server or cloud-based service (e.g., communicatively coupled to a user computer and / or a user interface). In some embodiments, the method 1000 includes the method 900. Any other method described herein can be used to facilitate or implement the method 1000. In some embodiments, not all of the blocks are computer-implemented (e.g., they can be user-implemented). Thus, computer-implemented portions of the method 1000 can refer to any blocks that can be implemented by a computer rather than a user.

[0115] The method 1000 is initialized at block 1002. Other blocks of the method 1000 are described below.

[0116] Data corresponding to radiographic imaging slices are obtained at block 1004 of the method 1000 (e.g., by causing an imaging device to obtain the slices). The slices can be received and loaded by a computer (e.g., the computer 414) for display on a user interface (e.g., the user interface 416).

[0117] Designing a virtual intervention path is now described.

[0118] In some embodiments, a scan is performed using an imaging device. The obtained data can be reconstructed to obtain (e.g., thin slice) MPR images, which can be loaded into an application (e.g., for visualization on a user interface). The application can be used to display axial, sagittal, and coronal MPR images.

[0119] The displayed images are used to view the lesion location at block 1006 of the method 1000. The user can select to enter the needle point and determine a virtual interventional path design.

[0120] A virtual path can be displayed and saved by the application at block 1008 of the method 1000. Any MPR images (e.g., sagittal tilt) can also include the virtual path of the needle.

[0121] Collimation to the imaging device can be selected according to the specified radiation dose at block 1010 of the method 1000 and can be based on the virtual needle path. If less radiation can be delivered by using the collimation function, the dose delivered to the subject can be reduced. In some cases, the width of the radiation delivered by the imaging device can be in the range of 2 to 4 cm.

[0122] Selecting the appropriate collimation range according to the virtual path and lesion location can improve the efficiency of the procedure. During the actual puncture procedure (as described below), the needle can be continuously probed and the same lesion location can be scanned multiple times.

[0123] As part of the verification procedure, additional radiographic imaging slices can be obtained at block 1012 of the method 1000 (e.g., corresponding to block 1004, but can depend on the collimation determined at block 1010). As part of this procedure, the real needle location can be determined as described below.

[0124] The real location of the needle is determined according to certain methods described herein at block 1014 of the method 1000. The real needle path and the virtual path can be displayed on the user interface.

[0125] Additional information can be determined according to the real location of the needle at block 1016 of the method 1000. For example, the distance between the needle tip and the lesion location and / or the deviation between the real needle trajectory and the trajectory of the virtual path.

[0126] If it is determined that the needle has not reached the lesion, the verification procedure can continue / repeat as determined at block 1018 of the method 1000. If appropriate, at block 1020, the user can cause the needle to be inserted further and / or change the trajectory of the needle in order to reach the lesion location.

[0127] At block 1022, once the needle has reached the lesion, the procedure can be considered complete.

[0128] In some embodiments, the method 1000 can have two parts. In the first part, the virtual path is determined and the needle is inserted to the virtual path. In the second part, the real needle location is determined and the needle is adjusted to reach the lesion. Figure 10In this regard, the first portion 1024 of the procedure (including blocks 1004-1010) includes a“planning” interventional procedure, and the second portion 1026 of the procedure (including blocks 1012-1020) includes a“verification” interventional procedure.

[0129] In implementing certain blocks of the method 1000, the interventional procedure can involve any of the following procedures.

[0130] In some embodiments, the position of the patient table can be determined relative to the position of the needle (see Figure 4 ), which can simplify the implementation of the interventional procedure. From the virtual puncture path, the method 1000 can include automatically calculating the position of the patient table at the needle tip position and the lesion position. Based on the saved position of the patient table, it can be determined which part of the subject’s body is being imaged, while also maintaining control of the position of the needle relative to the lesion position. Thus, in some embodiments, the patient table can be directly controlled according to the results of the calculation of the virtual path to guide the patient table to the specified position. A laser or other indicator can be used to indicate the entry point on the body (e.g., according to a laser positioning line). The user can then cause the needle to approach the entry point and insert into the body.

[0131] In some embodiments, the real needle trajectory can be automatically identified and displayed on the user interface. For example, after the needle is inserted, collimation can be used for multiple scans to reduce dose, and imaging slices can be automatically loaded into the application to enable determination of the needle trajectory. In some operations, the gantry and / or patient table angle can not be adjusted by physical tilting. Thus, the above-described digital tilting method can be used. Thus, the user can not need to adjust the gantry angle and / or the patient table angle. The method 1000 can automatically identify the needle trajectory and display a needle trajectory image, which can simplify the procedure for the user.

[0132] In some embodiments, relevant parameters can be automatically measured to simplify the workflow for the user. For example, when the needle is inserted, the angle of the needle can not be precisely controlled, resulting in a cross-layer phenomenon on the image. This can cause problems in confirming the needle position. The method 1000 can provide automatic identification of the needle position, the distance from the needle tip to the lesion position, the actual path, and the virtual design. Certain information, such as the deviation of the needle path angle (e.g., in combination with the predicted needle trajectory), can be displayed on the user interface. For example, the user can use this information to visually confirm the position of the needle in the tissue in order to facilitate active path adjustment at any time during the procedure. Thus, the needle can be accurately positioned at the lesion, which can facilitate accurate pathological diagnosis.

[0133] Figure 11A series of images are shown that can be displayed on the user interfaces described herein, e.g., when implementing the method 1000. The top row shows images in the axial plane, while the bottom row shows corresponding images in the sagittal plane. The images in the left column (labeled (a) and (b)) show virtual (planned) needle paths and real (tracked) needle paths in the axial and sagittal planes, respectively. The images in the center column (labeled (c) and (d)) show real (tracked) needle paths. The images in the right column (labeled (e) and (f)) show virtual (planned) needle paths. In Figure 11 In the middle, user guides in the images are represented with white "guide" lines. The white dashed lines refer to virtual or planned needle paths. The white solid lines refer to real or tracked needle paths. The "guide" lines can be colored when the images are displayed. For example, the planned needle paths can be shown as "blue," while the tracked needle paths can be shown as "green." Any color combination can be used. In addition, the lines can have different thicknesses and / or be dotted or dashed to provide guidance to the user. The various lines (or other markers) can be found useful by the user to guide them during the procedure when planning and performing an interventional procedure.

[0134] Figure 12 A device 1200 for visualizing a needle inserted in a body is shown. The device 1200 comprises a processing unit 1202 (which can correspond to the computer 414 of Figure 4 ) configured to implement certain methods described herein (to determine the position of the needle, for example). The device 1200 further comprises a display unit 1204 (which can correspond to the user interface 416 of Figure 4 ) configured to display images of the body superimposed with the position of the needle.

[0135] Figure 13 A tangible machine-readable medium 1300 is shown. The tangible machine-readable medium 1300 comprises instructions 1302 that, when run on at least one processor 1304, cause the at least one memory 1304 to implement certain methods described herein. In this embodiment, the instructions 1302 comprise instructions 1306 configured to implement block 102 of the method 100. The instructions 1302 further comprise instructions 1308 configured to implement block 104 of the method 100. Any of the methods described herein can be implemented with the aid of the tangible machine-readable medium 1300, which causes the at least one processor 1304 to implement such methods.

[0136] In some cases, any of the modules, processing circuits, or processing units (e.g., computer 414 and / or processing unit 1202) described above can comprise at least one special-purpose processor (e.g., a special-purpose application-specific integrated circuit (ASIC) and / or a field-programmable gate array (FPGA) and / or the like) configured to perform the functions of the module.

[0137] While the application has been illustrated and described in detail in the drawings and foregoing description, such illustration and description is to be considered illustrative or exemplary only; the application is not limited to the disclosed embodiments.

[0138] One or more features described in one embodiment can be combined with or substituted for features described in another embodiment. For example, the methods 100, 200, 300, 900, and 1000 can be modified based on the features described with respect to the system 400, the apparatus 1200, and / or the machine-readable medium 1300, and vice versa.

[0139] Embodiments in the present disclosure can be provided as methods, systems, or as a combination of machine-readable instructions and processing circuitry. Such machine-readable instructions can be included on, in, or with a non-transitory machine (e.g., computer) readable storage medium (including, but not limited to, disk storage, CD-ROMs, optical storage, and the like) having computer readable program code.

[0140] The present disclosure is described with reference to the flowcharts and block diagrams in accordance with embodiments of the present disclosure. Although the flowcharts illustrate a particular order of execution, the order of execution can differ from that which is described. Blocks described with reference to one flowchart can be combined with blocks described with reference to another flowchart. It will also be appreciated that each block of the flowcharts, and combinations of blocks in the flowcharts, can be implemented by machine readable instructions. Such machine readable instructions can be realized as machine code, bytecode, interpreted code, or as scripts, among others.

[0141] Machine readable instructions may, for example, be executed by a general purpose computer, a special purpose computer, an embedded processor, or a processor of other programmable data processing equipment to perform the functions described in the specification and diagrams. In particular, a processor or processing circuitry or modules thereof can execute machine readable instructions. Thus, the functional modules of the apparatuses (e.g., processing unit 1202) and other devices described herein can be implemented by a processor executing machine readable instructions stored in memory or operating in accordance with instructions embedded in logic circuitry. The term processor is to be interpreted broadly to include a CPU, processing unit, ASIC, logic unit, or programmable gate array, among others. The methods and functional modules can be performed by a single processor or divided among several processors.

[0142] Such machine readable instructions can also be stored in a computer readable storage device that can direct a computer or other programmable data processing apparatus to function in a particular manner,

[0143] Such machine readable instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus implement the functions specified in the flowchart block or blocks.

[0144] Furthermore, the teachings herein can be implemented in the form of a computer program product, the computer program product being stored in a storage medium and comprising instructions for making a computer device implement the methods recited in the embodiments of the present disclosure.

[0145] Elements or steps described in relation to one embodiment can be combined with or substituted for elements or steps described in relation to another embodiment. Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practising the claimed invention, from an study of the drawings, the disclosure, and the claims. The word "comprising" does not exclude other elements or steps and the word "one" or "a" does not exclude a plurality. A single processor or other unit can fulfil the functions of several items recited in the claims. While some measures are recited in dependent sub-claims, combination of these measures can also be used in advantage. A computer program can be stored / distributed on a suitable medium, such as an optical storage medium or a solid-state storage medium supplied together with or as part of other hardware, but can also be distributed in other forms, such as via the internet or other wired or wireless telecommunication systems. Any reference signs in the claims should not be construed as limiting the scope.

Claims

1. A computer program product comprising machine readable instructions which, when executed by a processor of a computer, cause the computer to perform a method comprising: receiving (102) data corresponding to a plurality of radiographic imaging slices of a body; determining (104) a position of needle insertion in the body based on combining information from at least one radiographic imaging slice comprising an indication of a first portion of the needle outside the body and at least another radiographic imaging slice comprising an indication of a second portion of the needle inside the body, wherein, determining the position of the needle comprises generating a combined needle region by merging data corresponding to a position of the first portion of the needle outside the body and data corresponding to a position of the second portion of the needle inside the body; and generating (106) display data for providing a visual representation of the needle in an image of the body, the visual representation in combination with a visual representation of at least the first portion and the second portion of the needle superimposed on the image, wherein the image is in a plane which is digitally tilted with respect to a plane parallel to the plurality of radiographic imaging slices.

2. The computer program product of claim 1, wherein, the method comprises causing (202) a user interface to display the image.

3. The computer program product of claim 1 or 2, wherein, determining the position of the needle comprises fitting a line to a plurality of regions in the received data indicative of the position of the first portion and the second portion of the needle.

4. The computer program product of claim 1 or 2, wherein, determining the position of the needle comprises determining a true needle region from a plurality of candidate needle regions by minimizing an energy function derived from the received data.

5. The computer program product of claim 4, wherein, the energy function is based on at least one of: a degree of dispersion of the candidate needle region; an area of the candidate needle region; an average pixel value of an edge of the candidate needle region; an average pixel value of an inner portion of the candidate needle region; an average of all cross-sectional values of the candidate needle region; a circularity parameter; and a bias parameter.

6. The computer program product of any one of claims 1, 2, and 5, wherein, the method comprises: identifying (306) the first portion of the needle outside the body by performing a morphological opening operation on the received data corresponding to the first portion of the needle; and determining a three-dimensional region corresponding to the first portion of the needle based on a planned path for the needle, wherein the three-dimensional region comprises a plurality of candidate needle regions from which a true needle region can be determined.

7. The computer program product of any one of claims 1, 2, and 5, wherein, the method comprises: identifying (308) the second portion of the needle inside the body by performing a threshold cut on the received data corresponding to the second portion of the needle; and determining a three-dimensional region corresponding to the second portion of the needle based on a planned path for the needle, wherein the three-dimensional region comprises a plurality of candidate needle regions from which a true needle region can be determined.

8. The computer program product of claim 7, wherein, the method comprises extending (310) the three-dimensional region corresponding to the second portion of the needle by: searching for at least one adjacent candidate needle region removed by the threshold cut and including data corresponding to the adjacent candidate needle region as part of the extended three-dimensional region.

9. The computer program product of any one of claims 1, 2, 5, and 8, wherein, the method comprises detecting (316) a tip of the needle based on a comparison of a measurement within a candidate needle tip region within the data to a threshold indicative of an absence of needle structure within the candidate needle tip region.

10. The computer program product of claim 9, wherein: if the comparison to the threshold indicates that a needle structure is present in the candidate tip region, then identifying another candidate tip region from the data to determine whether the another candidate tip region includes data indicative of a presence of a needle structure; and if the comparison to the threshold indicates that a needle structure is not present within the data corresponding to the candidate tip region, then a previously identified candidate tip region that includes data indicative of a presence of a needle structure within a previously identified candidate tip region is determined to contain the tip of the needle.

11. The computer program product of any one of claims 1, 2, 5, 8, and 10, wherein, The method includes detecting (318) a point of insertion of the needle on the body based on a first line fitted to a predicted trajectory of the needle, the predicted trajectory of the needle being determined based on the determined position of the needle and a second line fitted along a surface of the body.

12. The computer program product of any one of claims 1, 2, 5, 8, and 10, wherein, The method includes causing (320) a user interface to provide an indication of a difference between the predicted trajectory and a planned trajectory of the needle in response to determining that there is a deviation between the predicted trajectory and the planned trajectory of the needle.

13. An apparatus (400) for visualizing a needle (412) inserted in a body (402), the apparatus comprising: a processing unit (414) configured to implement a method performed by execution of the computer program product according to any of claims 1 to 12 to determine a position of the needle; and a display unit (416) configured to display an image of the body overlaid with the position of the needle.

14. A tangible machine-readable medium (1300) comprising instructions (1302) that, when executed on at least one processor (1304), cause the at least one processor to implement a method performed by execution of the computer program product according to any of claims 1 to 12.

Images