Determination of needle position

Abstract

To provide a method for determining a needle position during computed tomography guided intervention procedure.SOLUTION: A method 100 has a step 102 for receiving data corresponding to a plurality of imaging slices of a body; and a step 104 determining a position of a needle inserted in the body. Determination is based on combining information from at least one of the imaging slices including display of a first part of a needle outside the body and at least another imaging slice of the imaging slices including display of a second part of a needle inside the body. Combined needle areas are generated by combining data corresponding to the position of the first part of the needle outside the body with data corresponding to the position of the second part of the needle inside the body. The method further has a step 106 for generating display data to be provided in combination with visual expressions of the first part and the second part.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to methods, apparatus, and tangible machine-readable media for determining needle position, for example, during computed tomography (CT) guided interventional procedures. [Background technology]

[0002]

[0002] A subject's body is imaged using a CT imaging device, for example as part of an interventional procedure, such as a CT-guided needle biopsy, in which a needle is inserted into the body during imaging. A user, such as a physician or radiologist, needs to identify the location of the needle within the subject's body. Radiographic imaging slices acquired by the CT imaging device provide an indication of the location of the needle within the subject's body. For example, the radiographic imaging slices provide information regarding the needle direction, the location of the needle tip, and / or the needle entry point on the subject's skin surface. This information helps provide guidance to the user to determine the needle angle (or needle trajectory) offset from a planned needle path, the distance of the needle tip from an object, such as a lesion within the body, and / or the length of the needle already inserted.

[0003]

[0003] CT-guided needle biopsy of a lesion utilizes CT radiographic imaging slices, which provide detailed information for a user to make a specific decision regarding the biopsy. For example, CT radiographic imaging slices provide high-density and spatial resolution to pinpoint the exact location of the lesion and facilitate a clear understanding of the soft tissue conditions within and around the lesion, for example, to avoid certain structures or necrotic tissue.

[0004] In certain interventional procedures, a user acquires two-dimensional radiographic imaging slices acquired along an axial imaging plane to view a biopsy object on a user interface, the user relies on experience to plan an interventional path based on an analysis of multiple radiographic imaging slices acquired along the axial plane, and / or the user performs an interventional procedure involving a physical tilt in which the imaging device is tilted relative to the subject's body to acquire radiographic imaging slices along a specified plane that is different from the axial plane. Summary of the Invention [Problem to be solved by the invention]

[0005]

[0005] Planning and performing an interventional procedure is a relatively time-consuming task, since complete information about the needle position is not determined from a single radiographic imaging slice. For example, one radiographic imaging slice contains information about the tip of the needle, while another radiographic imaging slice contains information about another part of the needle. Thus, the actual intervention path spans multiple radiographic imaging slices. Therefore, it is not possible to display the entire intervention path in one image on the user interface. Therefore, the user relies on experience and / or acquires additional radiographic imaging slices to ensure that the needle position can be accurately determined. However, the additional scans take time and increase the radiation dose.

[0006]

[0006] Users who view one radiographic imaging slice at a time find it difficult and / or time-consuming to establish the information necessary to safely and / or reliably perform an interventional procedure because multiple radiographic imaging slices must be evaluated before deciding how to proceed with the interventional procedure. [Means for solving the problem]

[0007] Aspects or embodiments described herein relate to improving needle position determination and / or visualization within the body. The aspects or embodiments described herein avoid one or more problems associated with planning and performing interventional procedures using radiological imaging slices.

[0008]

[0008] In a first aspect, a method is described. The method is a computer-implemented method. The method includes receiving data corresponding to a plurality of radiographic imaging slices of a body. The method further includes determining a position of a needle inserted into the body. The determination is based on combining information from at least one of the radiographic imaging slices including a representation of a first portion of the needle outside the body and at least one other of the radiographic imaging slices including a representation of a second portion of the needle inside the body. Determining the needle position includes generating a combined needle region by merging data corresponding to a position of the first portion of the needle outside the body with data corresponding to a position of the second portion of the needle inside the body. The method further includes generating display data for providing a visual representation of the needle in an image of the body combined with a visual representation of at least the first and second portions of the needle superimposed on the image. The image is in a plane digitally tilted with respect to a plane parallel to the plurality of radiographic imaging slices.

[0009]

[0009] In some embodiments, the method includes causing a user interface to display an image.

[0010]

[0010] In some embodiments, determining the position of the needle includes fitting a line to a plurality of regions within the received data indicating the positions of the first and second portions of the needle.

[0011] In some embodiments, determining the needle location comprises determining an actual needle region from a plurality of candidate needle regions, the actual needle region being determined by minimizing an energy function derived from the received data.

[0012]

[0012] In some embodiments, the energy function is based on at least one of the discreteness of the candidate needle region, the area of ​​the candidate needle region, the average pixel value of the edge of the candidate needle region, the average pixel value of the interior of the candidate needle region, the average of all cross-sectional values ​​of the candidate needle region, a circularity parameter, and a deviation parameter.

[0013] In some embodiments, the method includes identifying a first portion of the needle outside the body by performing a morphological process opening on the received data corresponding to the first portion of the needle. The method further includes determining a three-dimensional region corresponding to the first portion of the needle based on a planned path of the needle. The three-dimensional region includes a plurality of candidate needle regions from which an actual needle region can be determined.

[0014] In some embodiments, the method includes identifying a second portion of the needle within the body by performing a threshold cutoff on the received data corresponding to the second portion of the needle. The method includes determining a three-dimensional region corresponding to the second portion of the needle based on a planned path of the needle. The three-dimensional region includes a plurality of candidate needle regions from which an actual needle region can be determined.

[0015] In some embodiments, the method includes expanding the three-dimensional region corresponding to the second portion of the needle. Expanding the three-dimensional region includes searching for at least one adjacent candidate needle region that is removed by threshold truncation. Expanding the three-dimensional region includes including data corresponding to the adjacent candidate needle region as part of the expanded three-dimensional region.

[0016]

[0016] In some embodiments, the method includes detecting a tip of the needle based on a comparison of measurements within a candidate needle tip region in the data to a threshold indicative of a lack of presence of a needle structure within the candidate needle tip region.

[0017] In some embodiments, if the comparison to the threshold indicates that a needle structure is present in the candidate needle tip region, another candidate needle tip region is identified from the data to determine whether the other candidate needle tip region includes data indicative of the presence of a needle structure. If the comparison to the threshold indicates an absence of a needle structure in the data corresponding to the candidate needle tip region, a previously identified candidate needle tip region that includes data indicative of the presence of a needle structure within the previously identified candidate needle tip region is determined to include the tip of a needle.

[0018] In some embodiments, the method includes detecting an insertion point of the needle on the body. The detecting the insertion point is based on a first line fitted to a predicted trajectory of the needle. The predicted trajectory of the needle is determined based on the determined position of the needle and a second line fitted along a surface of the body.

[0019]

[0019] In some embodiments, the method includes causing a user interface to provide an indication of a difference between the predicted trajectory and the planned trajectory of the needle. The indication of the difference is provided in response to a determination that there is a deviation between the predicted trajectory and the planned trajectory.

[0020]

[0020] In a second aspect, a method is described. The method is a computer-implemented method. The method includes receiving data corresponding to at least one radiographic imaging slice of the body, the data including information indicative of a position of a needle inserted into the body. The method further includes determining a predicted trajectory of the needle based on the information. The method further includes generating display data for providing a visual representation of the needle in an image plane parallel to a line including the predicted trajectory of the needle, the image plane being tilted relative to the at least one radiographic imaging slice of the body.

[0021]

[0021] In a third aspect, an apparatus is described. The apparatus is for visualizing a needle inserted into a body. The apparatus comprises a processing unit configured to perform the method of any aspect or embodiment of the first aspect and / or the second aspect. The apparatus determines a position of the needle. The apparatus further comprises a display unit configured to display an image of the body superimposed on the position of the needle.

[0022] In a fourth aspect, a tangible machine-readable medium is described. The tangible machine-readable medium includes instructions that, when executed on at least one processor, cause the at least one processor to perform a method of any aspect or embodiment of the first aspect and / or second aspect.

[0023]

[0023] These and other aspects of the invention will be apparent from and elucidated with reference to the embodiment(s) described hereinafter.

[0024]

[0024] Exemplary embodiments of the present invention will now be described, by way of example only, with reference to the following drawings, in which: [Brief description of the drawings]

[0025] [Figure 1]

[0025] A diagram illustrating a method for determining needle position according to an embodiment. [Diagram 2]

[0026] 1 illustrates a method for determining needle position according to an embodiment. [Diagram 3]

[0027] 1 illustrates a method for determining needle position according to an embodiment. [Figure 4]

[0028] 1 is a schematic diagram of a system for determining needle position according to an embodiment. [Figure 5a]

[0029] 1 is an image used in a particular method described in an embodiment. [Figure 5b] 1 is an image used in a particular method described in an embodiment. [Figure 6a]

[0030] 1 is an image for visualizing needle position according to an embodiment. [Figure 6b] 1 is an image for visualizing needle position according to an embodiment. [Figure 7a]

[0031] 1 is an image for visualizing needle position according to an embodiment. [Figure 7b] 1 is an image for visualizing needle position according to an embodiment. [Figure 8a]

[0032] 1 is an image for visualizing needle position, according to an embodiment. [Figure 8b] 4A-4D are corresponding diagrams for determining needle position, according to an embodiment. [Figure 9]

[0033] 1 illustrates a method for determining needle position according to an embodiment. [Figure 10]

[0034] 1 illustrates a method for determining needle position according to an embodiment. [Figure 11]

[0035] 1 is an image for visualizing needle position according to an embodiment. [Figure 12]

[0036] 1 is a schematic diagram of an apparatus for determining needle position according to an embodiment; [Figure 13]

[0037] 1 is a schematic diagram of a machine-readable medium for determining needle position according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0026]

[0038] 1 illustrates a method 100 (e.g., a computer-implemented method) for determining a needle position within a subject. Method 100 is implemented by a computer, such as a user computer communicatively coupled to a user interface, or a server or cloud-based service (e.g., communicatively coupled to a user computer and / or a user interface).

[0027]

[0039] The method 100 includes receiving data corresponding to a plurality of radiographic imaging slices of a body at block 102. The plurality of radiographic imaging slices are obtained by imaging a plurality of parallel cross-sections of the body with a CT imaging device. The CT imaging device is communicatively coupled to a computer that performs the method 100 to cause the CT imaging device to perform the imaging and / or to receive data from the CT imaging device. The computer that performs the method 100 may be separate from or the same as the computer used to construct the radiographic imaging slices for visualization by a user interface.

[0028]

[0040] The multiple radiographic imaging slices are parallel to each other. The radiographic imaging slices are acquired for a particular plane with respect to the body. For example, the radiographic imaging slices are acquired in an axial plane perpendicular to a long axis defined by the length of the body. The data acquired for the radiographic imaging slices corresponding to the axial plane (or another plane) is transformed via multiplanar reconstruction (MPR) to allow visualization of the body in a different plane, such as a sagittal plane, a coronal plane, or another plane. In any case, when generating data for reconstructing a visual representation of the body imaged by the imaging device, data from the multiple radiographic imaging slices is combined or merged to create a three-dimensional representation of the imaged portion of the body, which can be displayed on a user interface.

[0029]

[0041] The method 100 further includes determining a position of the needle inserted within the body based on combining information from at least one of the radiographic imaging slices including a view of the first portion of the needle outside the body and at least one other of the radiographic imaging slices including a view of the second portion of the needle within the body at block 104. Determining the needle position includes generating a combined needle region by merging data corresponding to the position of the first portion of the needle outside the body with data corresponding to the position of the second portion of the needle within the body.

[0030]

[0042] The method 100 further includes generating display data to provide a visual representation of the needle in the body image combined with a visual representation of at least the first and second portions of the needle superimposed on the image, at block 106. The image is in a plane that is digitally tilted relative to a plane parallel to the multiple radiographic imaging slices.

[0031]

[0043] When performing an interventional procedure, a needle is inserted into the body with a trajectory (i.e., direction) specified by a user. This trajectory is determined according to the user's experience according to a clinical scenario, for example to avoid certain tissues in the body. If radiographic imaging slices are acquired in an axial plane and the needle trajectory is tilted relative to the axial plane, different parts of the needle are detected in each of the radiographic imaging slices. In other words, a single radiographic slice does not contain all the information about the needle's position.

[0032]

[0044] By combining information from radiographic imaging slices about a first portion of the needle outside the body with information about a second portion of the needle inside the body, the position of the needle is determined. The needle position provides information about the length of the needle inserted into the body, the distance of the needle tip (or "point") from an object such as a lesion within the body, the trajectory (or direction) of the needle relative to a particular feature of the body (such as a particular tissue), and the entry point of the needle at the surface of the body.

[0033]

[0045] Because the information references both the first portion and the second portion and from which the needle trajectory can be determined (e.g., based on the relative difference in position between the first portion and the second portion), the needle position can be determined more easily or accurately than when displaying one radiographic imaging slice at a time (e.g., on a user interface).

[0034]

[0046] According to method 100, an image is generated using a digital tilt procedure, which is described in detail below in connection with FIG.

[0035]

[0047] 2 illustrates a method 200 (e.g., a computer-implemented method) for determining a needle position within a subject's body (used to facilitate visualization of the needle in an image). Method 200 is implemented by a computer, such as a user computer communicatively coupled to a user interface, or a server or cloud-based service (e.g., communicatively coupled to a user computer and / or a user interface). In some embodiments, method 200 is combined with method 100 of FIG. 1. Accordingly, blocks 102 through 106 of FIG. 1 are illustrated in FIG. 2 for ease of reference.

[0036]

[0048] In some embodiments, the method 200 includes, at block 202, displaying in a user interface an image of the body (e.g., from the display data generated at block 106 of the method 100) combined with a visual representation of at least a first portion and a second portion of the needle superimposed on the image (e.g., a sufficient length of the needle is displayed, such as the entire length of the needle, so that the user can determine the needle's trajectory). In some embodiments, the image displayed in the user interface includes an MPR image and / or a three-dimensional reconstruction of the body that can be optionally rotated according to the user's needs. Viewing the image, the user can decide how to proceed with the interventional procedure. Since more information about the needle's status is revealed from the image, the user can perform the interventional procedure more accurately, quickly, and / or efficiently than when viewing an image that includes partial information about the needle, such as when viewing individual radiographic imaging slices. The user interface is automatically updated as the interventional procedure continues in response to further data received from the imaging device.

[0037]

[0049] In some embodiments, determining the position of the needle includes merging data corresponding to a position of the first portion of the needle outside the body with data corresponding to a position of the second portion of the needle inside the body. The merged data is used to generate a combined needle region from the received data. The method 100 further includes determining the position of the needle using the combined needle region.

[0038]

[0050] In some embodiments, determining the position of the needle comprises fitting a line to a plurality of regions in the received data indicative of the positions of the first and second portions of the needle. Once the (relative) positions of the first and second portions can be established, it is possible to determine the trajectory of the needle. By fitting a line to the portions of the data indicative of the positions of the first and second portions, the trajectory of the needle is determined.

[0039]

[0051] 3 illustrates a method 300 (e.g., a computer-implemented method) for determining a position of a needle within a subject. Method 300 is implemented by a computer, such as a user computer communicatively coupled to a user interface, or a server or cloud-based service (e.g., communicatively coupled to a user computer and / or a user interface). In some embodiments, method 300 includes method 100 and / or method 200.

[0040]

[0052] The method 300 is initialized at block 302. Further blocks of the method 300 are described below.

[0041]

[0053] At block 304 of method 300, the user inputs a plan for the intervention procedure. For example, previous imaging by an imaging device and / or a physical examination of the body provides the user with information that allows the user to make a plan for how to perform the intervention procedure. The plan includes a target point (e.g., a lesion within the body) and an entry point on the surface of the body. If the entry point is accurately targeted, certain methods described herein provide sufficient information to allow the user to determine the trajectory of the needle as it is inserted into the body, allowing the user to determine whether a change in trajectory is necessary to reach the target point according to the plan. Furthermore, certain methods described herein provide information regarding the length of the needle inserted into the body and / or the location of the needle tip, allowing the user to determine how far the needle tip is from the target point.

[0042]

[0054] A first portion of the needle is detected at block 306 of method 300. A second portion of the needle is detected at method block 308. Blocks 306 and 308 may be performed in any order and may be performed based on data received from an imaging device (e.g., at block 102 of method 100).

[0043]

[0055] In some embodiments, block 306 of method 300 includes identifying a first portion of the needle outside the body by performing a morphological opening process (or any other suitable image processing algorithm) on the received data corresponding to the first portion of the needle (e.g., to extract the needle region outside the body and any other regions that appear to be needles). Method 300 further includes determining a three-dimensional region corresponding to the first portion of the needle based on a planned path of the needle. The three-dimensional region includes a number of candidate needle regions from which an actual needle region can be determined. For example, a region of interest is based on the planned path, and the region of interest is expanded into a volume of interest that includes a number of candidate needle regions (in the following description, each candidate needle region is labeled with an "i").

[0044]

[0056] A candidate needle region includes a region (e.g., area) of pixels in the data (from each radiographic imaging slice). A candidate needle region may or may not have a fixed size or shape. However, the specific size or shape requirements depend on the configuration of the CT imaging device. Of the multiple candidate regions, the region that most resembles a needle is determined as the "true needle region" via energy calculation, as described below.

[0045]

[0057] In some embodiments, candidate needle regions are filtered based on pixel values ​​in the data, since outside the body there is a large difference in pixel values ​​between data points corresponding to needles (which are metal) and other data points corresponding to other components. For example, filtering may involve comparing detected pixel values ​​to a threshold pixel value and ignoring any candidate needle regions that do not appear to correspond to the presence of a needle structure.

[0046]

[0058] In some embodiments, block 308 of method 300 includes identifying the second portion of the needle within the body by performing a threshold cutoff (or any other suitable image processing algorithm is performed) on the received data corresponding to the second portion of the needle. Method 300 further includes determining a three-dimensional region corresponding to the second portion of the needle based on the planned path of the needle. As referenced in block 306, the three-dimensional region includes a number of candidate needle regions from which an actual needle region can be determined.

[0047]

[0059] In some embodiments, the method 300 includes, at block 310, expanding the three-dimensional region corresponding to the second portion of the needle by searching for at least one adjacent candidate needle region that was removed by threshold truncation and including data corresponding to the adjacent candidate needle region as part of the expanded three-dimensional region.

[0048]

[0060] In some embodiments, determining the needle location comprises determining the actual needle region from a plurality of candidate needle regions by minimizing an energy function derived from the received data.

[0049]

[0061] In some embodiments, the energy function is based on at least one of the discreteness of the candidate needle region, the area of ​​the candidate needle region, the average pixel value of the edge of the candidate needle region, the average pixel value of the interior of the candidate needle region, the average of all cross-sectional values ​​of the candidate needle region, a circularity parameter, and a deviation parameter.

[0050]

[0062] In some embodiments, the circularity parameter is based on whether the candidate needle region is circular or not. In other words, the circularity parameter provides an indication as to whether the candidate needle region is circular by its size.

[0051]

[0063] In some embodiments, the deviation parameter is based on whether there is a deviation between the planned trajectory of the needle and the measured trajectory of the needle.

[0052]

[0064] A description is given of the calculation of the energy function and how it is used to determine the needle region. The calculation is performed in the two blocks 306 and 308 mentioned above.

[0053]

[0065] For block 306 (for the portion of the needle outside the body), in some embodiments the energy function is:

[0066]

number

[0067]

number

[0068]

number

[0069] α,β,γ,δ,ε,θ are the weights of each feature. The weights of each feature are fixed values ​​determined from experiments. e(i) is the discreteness of candidate needle region i. Discreteness refers to how the points of the region are distributed or collected into a line. If the points are evenly distributed in space, the value of e(i) is low, and if the points form a line, the value of e(i) is high. a(i),d(i),c(i) and n(i) are the weights of the cross sections (j∈{1,2,... ...N i}), where i refers to a 3D candidate needle region that contains multiple cross sections, and j refers to a cross section of this region. a(i) is the average circular area (i.e., of the candidate needle region), d(i) is the average pixel value of the edge (i.e., of the candidate needle region). n(i) is the average pixel value of the inner region (i.e., of pixels not on the edge of the candidate needle region). r(i) describes whether the area is circular or not (i.e., the "circularity parameter"). For region "i", c(i) is the average of the values ​​of all cross sections j. See, for example, the formula above. Where P jis the perimeter of the cross-sectional area j of a particular candidate needle region i, and S i and L i are the minor and major axes, respectively, of the region projected along the region direction pointing to the needle direction of the actual needle region (for other candidate needle regions, the region direction points to the longest direction of the region). v(i) is the deviation between the surface direction and the actual direction (i.e., the "deviation parameter").

number

number

number

number

[0054]

[0070] Once the energy function is determined, an optimization problem is performed to select the actual needle region from multiple candidate needle regions, i.e.

[0071] Get arg min E(i).

[0055]

[0072] The procedure for obtaining the actual needle region of the needle part inside the body is similar to the procedure for obtaining the actual needle region of the needle part outside the body. However, thresholding is performed instead of morphological processing opening when extracting the candidate needle region, and the energy function is different due to the difference in needle data between inside and outside the body. In some embodiments, the energy function is expressed as follows:

[0073]

number

[0056]

[0074] where the terms in the formula are as described above. In block 310, a neighborhood search is used to expand the needle region taking into account the previous threshold truncation process (which would otherwise result in missing certain regions of interest). Again, an optimization problem is run to obtain the actual needle region within the body.

[0057]

[0075] At block 312 of the method, the internal and external needle regions are combined (e.g., as referenced at block 104) to provide an accurate determination of the needle position (including, e.g., trajectory or direction). The accurate result is selected to represent the needle direction. For example, a comparison between the actual needle direction and the planned needle direction is used to determine the accuracy of the needle positioning.

[0058]

[0076] As part of the procedure to determine the needle position, and therefore the needle trajectory or direction, a line fitting is performed on the actual needle region in block 314. The coordinates of the detected region i can be expressed as:

[0077] {X,Y,Z}∈{(x i1 ,y i1 ,zx i1 ),... ...(x ij ,y ij ,z ij )},

[0059]

[0078] This means that the number of points in the candidate needle region i is j. By fitting a line using {X,Y,Z}, we can find a point (x0,y0,z0) on the needle and a unit direction vector (v x ,v y ,v z ) can be obtained. This information can be used to construct a visual representation of the needle position in the image displayed in the user interface.

[0060]

[0079] In some embodiments, the method 300 includes detecting the tip of the needle at block 316 based on a comparison of measurements within a candidate needle tip region in the data to a threshold value indicating the absence of a needle structure within the candidate needle tip region.

[0061]

[0080] In some embodiments, if the comparison to the threshold indicates that a needle structure is present in the candidate needle tip region, another candidate needle tip region is identified from the data to determine whether the other candidate needle tip region includes data indicative of the presence of a needle structure If the comparison to the threshold indicates an absence of a needle structure within the data corresponding to the candidate needle tip region, a previously identified candidate needle tip region that includes data indicative of the presence of a needle structure within the previously identified candidate needle tip region is determined to include the tip of a needle.

[0062]

[0081] Further description of the needle tip detection procedure is provided below.

[0063]

[0082] In some embodiments, the method 300 includes, at block 318, detecting an insertion point (e.g., an "entry point") on the body of the needle based on a first line fitted to a predicted trajectory of the needle determined based on the determined position of the needle and a second line fitted along the surface of the body.

[0064]

[0083] Further description of the insertion point detection procedure is given below.

[0065]

[0084] In some embodiments, the method 300 includes causing a user interface to provide an indication of the difference between the predicted and planned trajectories of the needle in response to determining that there is a deviation between the predicted and planned trajectories of the needle at block 320. For example, a user can identify deviations from the indication and, if appropriate, modify the intervention procedure to ensure that the target point is reached upon further insertion of the needle.

[0066]

[0085] At block 322, the method 300 may terminate or at least partially repeat the particular block of the method 300 (eg, once more data is received from the imaging device).

[0067]

[0086] 4 illustrates a system 400 for implementing certain methods described herein. System 400 is used to acquire radiological imaging slices as referenced in the methods and to perform certain blocks of the methods described herein.

[0068]

[0087] In the system 400, an interventional procedure is shown to be performed on a subject 402, and in particular on a particular part 404 of the subject's body (shown in dashed lines in FIG. 4). The subject is supported by a support 406, such as a couch. The position of the support 406 is controlled (e.g., by a computer) relative to an imaging device 408 (e.g., a CT imaging device) of the system 400. The imaging device 408 is used to acquire radiographic imaging slices. For example, the imaging device 408 includes an emitter (e.g., an X-ray emitter, not shown) and a corresponding detector (not shown), which are mounted on a gantry (forming part of the imaging device 408) that can rotate to acquire a series of measurements and acquire data for constructing each radiographic imaging slice. The imaging device 408 and the support 406 move relative to each other to acquire multiple radiographic imaging slices.

[0069]

[0088] An interventional procedure is performed in which a needle 410 (eg, supported by a needle holder 412) is inserted into the body of a subject 402 while radiological imaging slices are acquired.

[0070]

[0089] The system 400 further comprises a computer 414 (e.g., a processing unit comprising processing circuitry) for performing certain methods described herein. The computer 414 is communicatively coupled to a user interface 416, such as a display, for visualizing images acquired by the imaging device 408 and / or displaying information generated or determined by the computer 414.

[0071]

[0090] Although system 400 may appear to have imaging device 408, computer 414, and user interface 416 in the same location, in some cases these components may be in the same location or in different locations. For example, imaging device 408 may be in a different location than computer 414 and / or user interface 416. Computer 414 may be implemented by a user computer (e.g., connected to the same terminal as user interface 416) or may be implemented by a server or cloud-based service.

[0072]

[0091] Set forth below are some experimental images acquired according to certain methods or systems described herein. For ease of reference, certain features corresponding to features shown in system 400 are designated by reference numerals increased by 100 in Figures 5a and 5b, 200 in Figures 6a and 6b, and 300 in Figures 7a and 7b.

[0073]

[0092] 5a and 5b show different (CT) radiological imaging slices acquired by an imaging device (e.g., an imaging device as shown in FIG. 4) in an axial plane. 5a and 5b show the insertion of a needle 510 into the body of a subject 502. In 5a, a "first" part of the needle 510 is visible outside the body. In 5b, a "second" part of the needle 510 is visible inside the body. In this case, 5a refers to the seventh radiological imaging slice and 5b refers to the tenth radiological imaging slice from the multiple radiological imaging slices. Thus, a user needs to cycle between these different slices when trying to manually determine the position of the needle 510.

[0074]

[0093] Figures 6a and 6b show further images in which certain methods described herein have been implemented to allow visualization of the entire needle 610 within a single image. An entry point 630 on the surface of the body 602 and a needle tip location 632 are identified in Figures 6a and 6b to assist the user in performing the interventional procedure. A line 634 indicating the trajectory of the needle 610 is also shown in Figures 6a and 6b.

[0075]

[0094] Experimental data was obtained while performing interventional procedures on 31 patients (122 series) using certain methods described herein. The data was collected using various equipment (e.g., imaging devices) from various countries and included various body parts such as chest, lumbar, abdomen, shoulder, etc. The accuracy of determining needle location according to certain methods described herein was determined from this data. It was found that the accuracy of needle direction detection was 100%, the accuracy of needle tip detection was 91%, and the accuracy of entry point detection was 96%. Thus, certain methods described herein provide accurate determination of needle location (including, e.g., needle trajectory / direction, tip location, and entry point), which facilitates efficient and accurate performance of interventional procedures by a user.

[0076]

[0095] 7a and 7b show the implementation of needle tip detection on radiological imaging slices. The needle tip detection procedure is based on a segmentation 740 of the needle 710 (e.g., each slice includes a segment of the needle 710). Thus, each candidate needle region refers to a segment of the needle 710. Since the segmentation 740 does not necessarily identify the actual needle tip location, an additional procedure is used to identify the actual needle tip 742, as referenced in block 316.

[0077]

[0096] In some embodiments, if a needle structure is present in the candidate needle tip region, a (virtual) cylinder is created to detect further needle structures. This cylinder is aligned along the best fitting straight line 744 of the segmented needle part and includes the candidate needle tip region. From the current end of needle segmentation, it is determined whether there are still unsegmented needle structures within the cylinder. In other words, if the candidate needle tip region contains a needle structure but the adjacent candidate needle tip region (in front of the previously analyzed needle tip region) does not contain a needle structure, the previously analyzed needle tip region is considered to contain the needle tip, thereby locating the needle tip 742.

[0078]

[0097] Further description of needle tip characteristics is provided below.

[0079]

[0098] If a candidate needle tip region is identified as containing a needle structure, the method tracks along the needle direction (line 744). The pixel value difference is calculated between the central region 746 (i.e., the candidate needle tip region) and its surrounding regions 748 on each cross section of the cylinder. If the difference is large enough (meaning that the average CT pixel value of the central region 746 is much higher than that of the surrounding regions), it is determined that a needle structure still exists in that region. This difference is evident from the inset in FIG. 7a (which shows in enlarged form the portion of the figure including the needle tip 742). The inset in FIG. 7a shows the contrast between the central region 746 and the surrounding regions 748 in more detail. Thus, if a needle structure still exists in that region, tracking continues. If a needle structure is not detected (because there is less or no difference between the central region 746 and the surrounding regions 748), it is determined that the true needle tip 742 has been reached (because there is no needle structure in the previous region that indicates the needle tip has been reached).

[0080]

[0099] 8a and 8b show an implementation of needle entry point detection in relation to a radiological imaging slice. In this embodiment, needle entry point detection comprises fitting two lines to specific structures identified in the radiological imaging slice and identifying an intersection point of the two lines. The implementation of needle entry point detection in this embodiment is described below.

[0081]

[0100] The volume data is reconstructed perpendicular to the direction of the needle 810. A sagittal plane (similar to FIG. 8a, but other planes such as axial or coronal planes are used) of the reconstructed volume data includes the entire needle based on the three-dimensional coordinates of the needle 810. Thus, a line (e.g., a linear line) can be fitted to the reconstructed volume data.

[0082]

[0101] The surface of the body can be detected based on the orientation of the needle in the image. A line 850 (eg, a curve as shown in FIG. 8b) is fitted along the surface of the subject's body according to a quadratic curve equation.

[0083]

[0102] y=ax 2 +bx+c

[0084]

[0103] Here, it is assumed that the surface of the body in the sagittal plane can be represented by such an equation. In other cases, depending on the shape of the surface of the body, other equations, such as linear equations, are used.

[0085]

[0104] As mentioned above, the needle in the sagittal plane can be considered as a linear (2-dimensional) line 852, and the equation of the straight line is known based on the needle area according to the linear equation.

[0086]

[0105] y=kx+d

[0087]

[0106] By combining the above equations, it is possible to determine the intersection point corresponding to the needle entry point 854. In some cases, the entry point needs to be fine-tuned by further image processing analysis, as the body surface may not be smooth and / or may not correspond to the quadratic equation.

[0088]

[0107] The above methods and systems refer to a method for determining needle position. Another method for determining needle position is described below and implemented by the system 400 of FIG.

[0089]

[0108] 9 illustrates a method 900 (e.g., a computer-implemented method) for determining a needle position within a subject's body and / or facilitating visualization of a needle position. Method 900 is implemented by a computer, such as a user computer communicatively coupled to a user interface, or a server or cloud-based service (e.g., communicatively coupled to a user computer and / or a user interface). Method 900 may be implemented as part of or in conjunction with any other method described herein.

[0090]

[0109] Method 900 includes receiving data corresponding to at least one radiographic slice of a body, the data including information indicative of a position of a needle inserted into the body, at block 902. The data received at block 902 corresponds to the data received at block 102 of method 100.

[0091]

[0110] The method 900 includes determining a predicted trajectory of the needle based on the information, at block 904. The predicted trajectory may be determined using certain methods described herein, such as using method 100.

[0092]

[0111] The method 900 includes, at block 906, generating display data for providing a visual representation of the needle in an image plane parallel to a line containing the predicted trajectory of the needle. The image plane is tilted with respect to at least one radiographic imaging slice of the body. Thus, the user selects an image view at any angle (e.g., based on an MPR image or a three-dimensional reconstruction of the body) and the needle is shown completely in the image. Since certain methods described herein (e.g., method 100) allow the position of the needle (and thus its direction, tip position, and entry point) to be determined, it is possible to include a visual representation of the needle in any image location selected by the user. If the needle is at an angle with respect to the plane of the radiographic imaging slice (e.g., in an axial plane), it is possible to digitally tilt the image in order to visualize the entire needle within the body. For example, the image (corresponding to the display data) is in a plane that is digitally tilted with respect to a plane parallel to at least one radiographic imaging slice. In other words, the display data generated for the image is such that it is possible to visualize at least a part of the needle in a digitally reconstructed plane at an angle with respect to the plane of the radiographic imaging slice. The tilt / angle is changed depending on the user needs, which eliminates the need to perform a physical tilt to tilt the imaging device gantry to align with the needle in order to image the entire needle, which saves time.

[0093]

[0112] Instead of the physical tilt procedure, a digital tilt procedure is used. The digital tilt facilitates needle visualization in a specified or preferred direction / plane, which provides a flexible, intuitive and / or convenient needle visualization. Furthermore, a highly accurate (i.e., precise) angle and depth of the needle is determined using the digital tilt procedure, which perfectly represents the actual needle path during the interventional procedure. The path can be adjusted at any time and the scanning range can be precisely positioned to control the amount of scanning radiation irradiated by the imaging device, so that accurate sampling can be performed to create a basis for pathological diagnosis of the lesion, and also reduce unnecessary radiation damage to the subject.

[0094]

[0113] The digital oblique procedure involves planning and confirming an interventional procedure (i.e., lesion location and needle path design). It automatically identifies the actual needle path and presents a path image to confirm the current needle position, direction, and needle angle. Such information helps to determine the needle tip distance from the lesion location.

[0095]

[0114] FIG. 10 illustrates a method 1000 (e.g., a computer-implemented method) of determining a needle position within a subject. The method 1000 is implemented by a computer, such as a user computer communicatively coupled to a user interface, or a server or cloud-based service (e.g., communicatively coupled to a user computer and / or a user interface). In some embodiments, the method 1000 includes the method 900. Reference is made to any of the other methods described herein that are used to facilitate or implement the method 1000. In some embodiments, not all of the blocks are computer-implemented (e.g., they are user-implemented). Thus, the computer-implemented portion of the method 1000 refers to any block that can be implemented by a computer rather than a user.

[0096]

[0115] The method 1000 is initialized at block 1002. Further blocks of the method 1000 are described below.

[0097]

[0116] At block 1004 of method 1000, data corresponding to radiographic imaging slices is acquired (e.g., by having an imaging device acquire the slices), which are received and loaded by a computer (e.g., computer 414) for display on a user interface (e.g., user interface 416).

[0098]

[0117] Here, the design of a virtual intervention pathway is described.

[0099]

[0118] In some embodiments, a scan is performed using an imaging device. The acquired data can be reconstructed to obtain (e.g., thin slice) MPR images, which are loaded into an application (e.g., for visualization on a user interface). The application is used to display axial, sagittal, and coronal MPR images.

[0100]

[0119] The displayed image is used to observe the location of the lesion at block 1006 of method 1000. The user selects the entry needle point and determines the virtual intervention path design.

[0101]

[0120] The virtual path is displayed and saved by the application at block 1008 of the method 1000. The MPR images (e.g., sagittal oblique) also include the virtual path of the needle.

[0102]

[0121] At block 1010 of method 1000, the collimation of the imaging device is selected according to the specified radiation dose and based on the virtual needle path. Using the collimation function, if less radiation can be delivered, the dose delivered to the subject is reduced. In some cases, the width of the radiation delivered by the imaging device ranges from 2 to 4 cm.

[0103]

[0122] Selecting an appropriate collimation range according to the virtual path and the lesion location improves the efficiency of the procedure. During the actual puncture process (described below), the needle is continuously probed and the same lesion location is scanned multiple times.

[0104]

[0123] As part of the verification procedure, at block 1012 of method 1000, a further radiographic slice is acquired (e.g., corresponding to block 1004, but dependent on the collimation determined at block 1010). As part of this procedure, the actual needle position is determined, as described below.

[0105]

[0124] At block 1014 of method 1000, the actual position of the needle is determined according to certain methods described herein. The actual needle path and the virtual path are displayed on a user interface.

[0106]

[0125] At block 1016 of method 1000, further information is determined from the actual position of the needle, such as the distance between the needle tip and the lesion location and / or the deviation between the actual needle trajectory and the trajectory of the virtual path.

[0107]

[0126] The verification procedure continues / repeats if it is determined that the needle has not yet reached the lesion, as determined at block 1018 of method 1000. If appropriate, the user at block 1020 further inserts the needle and / or changes the needle trajectory to reach the lesion location.

[0108]

[0127] Once the needle reaches the lesion, block 1022, the procedure is considered complete.

[0109]

[0128] In some embodiments, there were two parts to the method 1000. In Figure 10, a first part 1024 of the procedure (including blocks 1004 to 1010) involves "planning" the interventional procedure, and a second part 1026 of the procedure (including blocks 1012 to 1020) involves "verifying" the interventional procedure.

[0110]

[0129] While performing a particular block of the method 1000, the intervention procedure may include any of the following procedures.

[0111]

[0130] In some implementations, the position of the couch (see FIG. 4) is determined relative to the needle position, which simplifies the implementation of the intervention procedure. According to the virtual puncture path, the method 1000 comprises automatically calculating the needle tip position and the position of the couch at the lesion position. Based on the saved position of the couch, it is possible to determine which part of the subject's body is being imaged while maintaining control of the needle position relative to the lesion position. Thus, in some embodiments, the couch is directly controlled to be directed to a specified position according to the calculation result of the virtual path. Using a laser or other pointer, the entry point of the body is indicated (e.g., according to a laser positioning line). The user then approaches the needle to the entry point and inserts it into the body.

[0112]

[0131] In some implementations, the actual needle trajectory is automatically recognized and displayed on the user interface. For example, after the needle is inserted, the collimation function can be used for multiple scans to reduce the dose and the imaging slices can be automatically loaded into the application to determine the needle trajectory. In some operations, the gantry and / or couch angle cannot be adjusted by physical tilt. Therefore, the digital tilt method described above is used. Thus, the user does not need to adjust the gantry angle and / or couch angle. The method 1000 automatically recognizes the needle trajectory and displays the needle trajectory image, thereby simplifying the user's procedure.

[0113]

[0132] In some implementations, relevant parameters can be measured automatically to simplify the user's workflow. For example, when the needle is inserted, the needle angle may not be precisely controlled, resulting in cross-layer phenomena in the image. This creates problems in confirming the needle position. The method 1000 provides automatic identification of the needle position, the distance from the needle tip to the lesion position, the actual path, and the virtual design. Specific information such as needle path angle deviation (e.g., combined with the predicted needle trajectory) is displayed in the user interface. This information can be used by the user to visually confirm the needle position in the tissue, for example, to facilitate active path adjustment at any time during the procedure. Thus, the needle can be precisely placed in the lesion, which facilitates accurate path diagnosis.

[0114]

[0133] FIG. 11 shows a series of images as displayed in a user interface described herein when performing method 1000. The top row shows images in the axial plane, while the bottom row shows corresponding images in the sagittal plane. The images in the left column (labeled (a) and (b)) show the virtual (planned) and actual (tracked) needle paths in the axial and sagittal planes, respectively. The images in the middle column (labeled (c) and (d)) show the actual (tracked) needle path. The images in the right column (labeled (e) and (f)) show the virtual (planned) needle path. In FIG. 11, the user's guide lines in the images are represented by white "guideline" lines. The dashed white lines refer to the virtual or planned needle path. The solid white lines refer to the actual or tracked needle path. When the images are displayed, the "guide" lines are colored. For example, the planned needle path is shown in "blue" and the tracked needle path is shown in "green". Any combination of colors may be used. Furthermore, the lines may be of different thickness and / or dotted or dashed to provide guidance to the user. When planning and performing an interventional procedure, the user will find various lines (or other markers) useful to guide them during the procedure.

[0115]

[0134] 12 shows an apparatus 1200 for visualizing a needle inserted into a body. The apparatus 1200 comprises a processing unit 1202 (corresponding to computer 414 in FIG. 4) configured to perform certain methods described herein (e.g., to determine the location of the needle). The apparatus 1200 further comprises a display unit 1204 (corresponding to user interface 416 in FIG. 4) configured to display an image of the body superimposed on the location of the needle.

[0116]

[0135] 13 illustrates a tangible machine-readable medium 1300. The tangible machine-readable medium 1300 includes instructions 1302 that, when executed on at least one processor 1304, cause the at least one processor 1304 to perform certain methods described herein. In this embodiment, the instructions 1302 include instructions 1306 configured to perform block 102 of the method 100. The instructions 1302 further include instructions 1308 configured to perform block 104 of the method 100. Any of the methods described herein may be performed by the tangible machine-readable medium 1300, which causes the at least one processor 1304 to perform such methods.

[0117]

[0136] In some cases, any of the above-mentioned modules, processing circuitry, or processing units (e.g., computer 414 and / or processing unit 1202) comprise at least one dedicated processor (e.g., an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), etc.) for performing the functions of the module.

[0118]

[0137] The invention has been illustrated and described in detail in the drawings and foregoing description, and such illustration and description are to be considered illustrative or exemplary and not restrictive, but the invention is not limited to the disclosed embodiments.

[0119]

[0138] One or more features described in one embodiment may be combined with or substituted for features described in another embodiment, for example, methods 100, 200, 300, 900, and 1000 may be modified based on features described in connection with system 400, apparatus 1200, and / or machine-readable medium 1300, and vice versa.

[0120]

[0139] Embodiments of the present disclosure may be provided as a method, a system, or a combination of machine-readable instructions and processing circuitry. Such machine-readable instructions may be contained in a non-transitory machine (e.g., computer) readable storage medium (including, but not limited to, disk storage devices, CD-ROMs, optical storage devices, etc.) having computer-readable program code therein or in the machine-readable instructions.

[0121]

[0140] The present disclosure is described with reference to flowcharts and block diagrams of methods, devices, and systems according to embodiments of the present disclosure. Although the above flowcharts show a specific order of execution, the order of execution may differ from that shown. Blocks described in relation to one flowchart may be combined with blocks of another flowchart. It should be understood that each block in the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, may be implemented by machine-readable instructions.

[0122]

[0141] The machine-readable instructions are executed by, for example, a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to implement the functions described in the description and drawings. In particular, the processor or processing circuitry, or modules thereof, executes the machine-readable instructions. Thus, the functional modules of the apparatus (e.g., processing unit 1202) and other devices described herein are implemented by a processor that executes machine-readable instructions stored in a memory, or operates according to instructions embedded in logic circuitry. The term "processor" is to be interpreted broadly to include a CPU, a processing unit, an ASIC, a logic unit, or a programmable gate array, etc. All of these methods and functional modules may be executed by a single processor or may be divided among several processors.

[0123]

[0142] Such machine-readable instructions are stored in computer-readable storage devices that can cause a computer or other programmable data processing device to operate in a particular mode.

[0124]

[0143] Such machine-readable instructions are loaded into a computer or other programmable data processing device such that the computer or other programmable data processing device performs a sequence of operations to generate computer-implemented processes such that the instructions executed on the computer or other programmable device implement the functions specified by the flowchart and / or block diagram blocks.

[0125]

[0144] Furthermore, the teachings herein may be embodied in the form of a computer program product, the computer program product being stored on a storage medium and including a plurality of instructions for causing a computing device to perform the methods detailed in the embodiments of the present disclosure.

[0126]

[0145] An element or step described in connection with one embodiment may be combined with or substituted for an element or step described in connection with another embodiment. Other variations to the disclosed embodiments will be understood and effected by those skilled in the art practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims. In the claims, the term "comprises" does not exclude other elements or steps, and the singular does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. A computer program may be stored or distributed by a suitable medium, such as an optical storage medium or a solid-state medium, provided together with or as part of other hardware, but also distributed in other forms, such as via the Internet or other wired or wireless communication systems. Reference signs in the claims should not be construed as limiting the scope.

Claims

1. 1. A computer-implemented method comprising: receiving data corresponding to a plurality of radiographic slices of the body; determining a position of the needle inserted within the body based on combining information from at least one of the radiographic imaging slices including a representation of a first portion of the needle outside the body and at least one other of the radiographic imaging slices including a representation of a second portion of the needle within the body, the determining step comprising: generating a combined needle region by merging data corresponding to the position of the first portion of the needle outside the body with data corresponding to the position of the second portion of the needle within the body; generating display data to provide a visual representation of the needle in the body image combined with a visual representation of at least the first and second portions of the needle superimposed on the image; The method, wherein the image is in a plane that is digitally tilted relative to a plane parallel to the plurality of radiographic imaging slices.

2. The method of claim 1 , further comprising causing a user interface to display the image.

3. 3. The method of claim 1 or 2, wherein determining the position of the needle comprises fitting lines to a plurality of regions within the received data that indicate the positions of the first and second portions of the needle.

4. 4. The method of claim 1, wherein determining the location of the needle comprises determining an actual needle region from a plurality of candidate needle regions by minimizing an energy function derived from the received data.

5. The method of claim 4 , wherein the energy function is based on at least one of a discreteness of the candidate needle region, an area of ​​the candidate needle region, an average pixel value of an edge of the candidate needle region, an average pixel value of an interior of the candidate needle region, an average of all cross-sectional values ​​of the candidate needle region, a circularity parameter, and a deviation parameter.

6. identifying the first portion of the needle outside the body by performing a morphological process opening on the received data corresponding to the first portion of the needle; determining a three-dimensional area corresponding to the first portion of the needle based on a planned path of the needle; The method of claim 1 , wherein the three-dimensional region includes a plurality of candidate needle regions from which the actual needle region can be determined.

7. identifying the second portion of the needle within the body by performing a threshold cutoff on the received data corresponding to the second portion of the needle; determining a three-dimensional area corresponding to the second portion of the needle based on a planned path of the needle; The method of claim 1 , wherein the three-dimensional region includes a plurality of candidate needle regions from which the actual needle region can be determined.

8. 8. The method of claim 7, further comprising: searching for at least one adjacent candidate needle region removed by the threshold truncation and expanding the three-dimensional region corresponding to the second portion of the needle by including data corresponding to the adjacent candidate needle region as part of the expanded three-dimensional region.

9. 9. The method of claim 1, further comprising detecting the tip of the needle based on a comparison of measurements within a candidate needle tip region in the data to a threshold that indicates the absence of a needle structure within the candidate needle tip region.

10. if the comparison with the threshold indicates that a needle structure is present in the candidate needle tip region, another candidate needle tip region is identified from the data and it is determined whether the other candidate needle tip region includes data indicative of the presence of a needle structure; 10. The method of claim 9, wherein a previously identified candidate needle tip region that includes data indicating the presence of a needle structure within the previously identified candidate needle tip region is determined to include the tip of the needle if the comparison with the threshold indicates a lack of a presence of a needle structure within the data corresponding to the candidate needle tip region.

11. 11. The method of claim 1, further comprising detecting an insertion point of the needle on the body based on a first line fitted to a predicted trajectory of the needle determined based on the determined position of the needle and a second line fitted along a surface of the body.

12. 12. The method of claim 1, further comprising, in response to determining that there is a deviation between the predicted and planned trajectories, causing a user interface to provide an indication of the difference between the predicted and planned trajectories of the needle.

13. 1. An apparatus for visualizing a needle inserted into a body, the apparatus comprising: A processing unit for carrying out the method according to any one of claims 1 to 12 in order to determine the position of the needle; and a display unit that displays an image of the body superimposed on the location of the needle.

14. A tangible, machine-readable medium comprising instructions that, when executed on at least one processor, cause the at least one processor to perform the method of any one of claims 1 to 12.

Images