Automatic and precise positioning of sliding tissue in medical images

Abstract

An image processing system (SYS) based on image sliding interface detection and related methods. The system comprises an input port (IN) for receiving two medical input images (I1, I2), the two medical input images are previously acquired for a patient (PAT) at different times during a sliding movement of at least one of at least two anatomical features (AF1, AF2) of the patient (PAT) relative to the other of the at least two anatomical features (AF1, AF2). The sliding motion defines a sliding interface (SI). A sliding motion evaluator (SME) applies a registration algorithm to calculate a series of metric maps configured to respond to sliding motion by changing values of algorithm parameters. A sensitivity analyzer (SA) calculates a sensitivity map representing a change in the series of metric maps according to the value of the parameter. A sliding interface detector (SD) detects an image-based position of the sliding interface representation based on the sensitivity map.

Description

Technical Field

[0001] The invention relates to an image processing system for image-based sliding interface detection, an arrangement comprising such a system, a related method, a computer program element and a computer-readable medium. Background Art

[0002] Certain anatomical structures in the human body may give rise to one or more sliding interfaces along which anatomical features of interest pass each other in sliding motion. Examples may include the motion of a lung lobe along a fissure surface or along the pleura, or the motion of a joint, etc.

[0003] Precise localization of such sliding interfaces (which may form surfaces or sliding layers) may be beneficial in supporting specific image processing tasks in diagnosis or therapy. Examples may include measuring articular cartilage thickness, analyzing lung function, aligning respiratory phase gated images, planning radiation therapy treatments, etc.

[0004] For example, one method of locating such a sliding interface may rely on segmentation applied to a single image volume. Segmentation may use image intensity or structures contained in the image. Although sometimes showing good results, imaging or reconstruction artifacts or anatomical changes caused by disease or even by poor image contrast may cause segmentation to fail or be inaccurate.

[0005] An alternative approach may use two or more images recorded while such sliding motion occurs, such as image acquisition of the knee joint during flexion / extension, or image acquisition of the lungs during the respiratory cycle. Another example is 4D-CT acquisition, where each scan is performed at a different respiratory phase. A related approach is described by R Amelon et al. in “A measure for characterizingsliding on lung boundaries” published as Ann. Biomed. Eng., vol 42(3), pp 642–50, (2014), in which image registration techniques are used. However, the described method requires prior knowledge of specific image registration parameters, which are usually unknown. Summary of the invention

[0006] Therefore, there may be a need for improved image processing.

[0007] The objects of the invention are solved by the subject-matter of the independent claims, wherein further embodiments are incorporated in the dependent claims.It should be noted that the aspects of the invention described below apply equally to the related method, medical imaging arrangement, computer program element and computer readable medium.

[0008] According to a first aspect of the present invention, there is provided an image processing system for image-based sliding interface detection, comprising:

[0009] an input port for receiving at least two medical input images acquired at different times for a patient during a sliding motion of at least one of at least two anatomical features of the patient relative to at least another of the at least two anatomical features, the sliding motion defining a sliding interface with respect to the at least two anatomical features;

[0010] a sliding motion evaluator configured to apply a registration algorithm to the at least two images, and to apply it with varying values ​​of parameters of the registration algorithm, to compute a series of metric maps, the series of metric maps being configured to be responsive to sliding motions recorded at the at least two input images;

[0011] a sensitivity analyzer configured to compute a sensitivity map representing changes in values ​​of the series of metric maps according to the changes in the parameter; and

[0012] A sliding interface detector is configured to detect a position of a representation of the sliding interface in at least one of the two input images based on the sensitivity map.

[0013] According to one embodiment, the parameter is a regularization parameter configured to influence a smoothing term based on which the algorithm is operable.

[0014] According to one embodiment, the registration algorithm is based on establishing a per-parameter vector field for at least one of the at least two images.

[0015] According to an embodiment, said metric is based on said vector field.

[0016] According to an embodiment system, the vector field is a deformation vector field.

[0017] According to an embodiment, the metric is based on any of: i) the difference between the vector fields, ii) the Jacobian matrix of the vector fields, iii) the eigenvalues ​​of the vector fields, (iv) an operator acting on the vector fields.

[0018] According to one embodiment, the system includes a visualizer configured to generate a graphical display for display on a display device, the graphical display including an indication of a location of a representation of the sliding interface in at least one of the at least two images.

[0019] According to one embodiment, the at least two images are acquired by any one of: i) an imaging device configured for transmission imaging, ii) an imaging device configured for emission imaging, iii) a magnetic resonance imaging device, iv) an ultrasound imaging device.

[0020] According to one embodiment, the detection operation performed by the sliding interface is based on threshold processing.

[0021] In another aspect, an arrangement is provided comprising a system according to any one of the preceding claims and an imaging device.

[0022] In another aspect, a computer-implemented method for image-based sliding interface detection is provided, comprising:

[0023] receiving at least two medical input images acquired of a patient at different times during a sliding motion of at least one anatomical feature of at least two anatomical features of the patient relative to at least another anatomical feature of the at least two anatomical features, the sliding motion defining a sliding interface with respect to the at least two anatomical features;

[0024] applying a registration algorithm to the at least two images, and at varying values ​​of parameters of the registration algorithm, to compute a series of metric maps, the series of metric maps being configured to respond to sliding motions recorded in the at least two input images;

[0025] calculating a sensitivity map representing changes in values ​​of the series of metric maps according to changes in the parameter; and

[0026] A position of a representation of the sliding interface in at least one of the at least two input images is detected based on the sensitivity map.

[0027] In a further aspect, a computer program element is provided which, when run by at least one computing unit, is adapted to perform the method.

[0028] In a further aspect, a computer readable medium having the program element stored thereon is provided.

[0029] The proposed system and method allow for fully automatic sliding interface detection. No user input is required. Furthermore, no segmentation is required, thereby overcoming the above-mentioned limitations of segmentation-based methods. Unlike many, if not all, segmentation methods, the system and method are not anatomy-specific and can be used to detect any such "anatomy-to-anatomy" sliding interface. Therefore, the proposed setup is invariant to the anatomy type. With the proposed system and method, no prior segmentation of the input image is required. Therefore, the system and method can be used without any segmentation.

[0030] The proposed system and method rely on motion changes / position changes rather than the anatomical structure itself. Since the processing uses multiple input images (rather than using a single image for segmentation), the proposed system and method are more robust.

[0031] In one embodiment, the system is configured to apply the registration algorithm in an iterative manner as a tool to study the sensitivity of a particular metric to changes in input parameter space. The derivative / gradient of the metric with respect to the changing input parameter can be used for sliding interface / sliding motion detection.

[0032] Using the determined shear amount for the local sliding metric proposed by Amelon et al. mentioned above requires that the weight of the regularization term in the registration algorithm be selected from an optimal range, which may not be straightforward: choosing a relatively low weight will overestimate the sliding, resulting in a large number of false positive results, which is undesirable. But choosing a relatively high weight will result in a fairly rigid deformation with little or zero sliding findings. The optimality of finding the best range for the regularization weight depends on the data set - both on the amplitude of the motion and on the affected area within the image. For example, in an abdominal scan, sliding along the pleura is easier to identify than sliding along the lung fissure or along the abdominal cavity. The proposed system and method avoid making such a selection and strike the right balance by monitoring the metric changes that vary with changes in the parameter dM / dP. Therefore, what is proposed in this article is to monitor changes in the range of the amount of deformation. By construction, however, there are values ​​within this range for which the response is optimal.

[0033] In general, the regularization weights cannot be chosen independently of the image alignment, since the regularizer is preferably weighted appropriately to ensure accurate alignment.

[0034] The proposed system and method avoids the need to select an optimal range for the regularizer weights. Instead, the proposal provides a general and robust method to fully automatically detect sliding motion or sliding interfaces.

[0035] The use of a (non-rigid) image registration algorithm is merely a tool in this context. The purpose of such algorithms is usually to provide a transformation in order to align a source image with a target image in a common coordinate system. However, the transformation computed by such an image registration algorithm is of little or no interest in this context. What is of interest here is the (deformed) vector field computed in the context of such an image registration algorithm. It is the behavior of such a (deformed) vector field that is utilized herein for the purpose of preferably fully automatic sliding interface detection.

[0036] More than two input images may be used, for example in conjunction with a group-by-group registration algorithm. Alternatively, when three or more images are used, concatenation and inversion may be used. For example, given input images A, B, and C, A may be registered to B and A to C. Vector fields for other combinations (e.g., C to B) may then be derived via concatenation and / or inversion of the vector fields.

[0037] Control point based registration can be used when the vector field does not need to be recalculated from scratch for every image location. A sparse set of control points can be used instead, and the local vector field for any desired voxel or pixel can be interpolated / extrapolated as needed.

[0038] A "user" refers to a person who operates the imaging device or oversees the imaging process, such as a medical staff member or other person. In other words, a user is typically not a patient. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Exemplary embodiments of the present invention will now be described with reference to the following drawings, which, unless otherwise indicated, are not to scale, and in which:

[0040] Figure 1 A schematic block diagram of a medical imaging arrangement is shown;

[0041] Figure 2 It shows that according to Figure 1 A block diagram of one embodiment of a system for image-based detection of sliding interfaces between anatomical features;

[0042] Figure 3A -C shows Figure 2 An embodiment of the system in; and

[0043] Figure 4 A flow chart of a computer-implemented method for image-based detection of sliding interfaces between anatomical features is shown. DETAILED DESCRIPTION

[0044] refer to Figure 1 , shows a schematic block diagram of a medical imaging arrangement. The arrangement may include a medical imaging device IA, which provides an input image to an image-based detection system SYS. The system is configured to detect one or more sliding interfaces IF specific to two or more anatomical features AF1, AF2 of a patient PAT captured in the image based on the input image. At the time of acquisition, this online, real-time configuration of the system SYS is preferred, in which the images are streamed to the system. However, this does not exclude other offline types of embodiments, in which the acquired images are first stored in a non-volatile memory and can then be retrieved therefrom by the system SYS (e.g., upon user request) for processing. The system can be implemented by one or more data processing units CU.

[0045] In more detail, the system SYS is implemented by a computer for processing input images I1, I2, comprising images (frames) of a patient's PAT acquired at different times, while there is a sliding motion relative to at least two such anatomical features AF1, AF2. The features I1, I2 may be related to the configuration of internal anatomical structures within the patient's PAT. The input images include at least one earlier image and at least one later image, designated herein as image / frame I1 and image / frame I2, respectively. There may be more than two such input images Ij, forming a sequence of images of a patient's region of interest, but in the following we will refer to two such input images I1, I2 without limiting the present disclosure.

[0046] The image is collected by a medical imaging device IA, which can be any modality suitable for the current imaging task, such as radiographic X-ray, tomographic X-ray, MRI, PET, SPECT or any other device. The input images I1 and I2 have a specific size in terms of pixels (such as columns and rows) arranged in a matrix structure. The sizes of the two images can be the same, and if they are not the same, they can be made the same by sampling. However, the two input images I1 and I2 in this article do not necessarily have the same size. The image is preferably 3D, such as a reconstructed 3D volume, but in some alternative embodiments, 2D projection images are not excluded as input images.

[0047] The detection system SYS is computer implemented for processing the input images I1, I2 so as to generate as an output an indication relating to an interface IF along which there is a relative motion between two (or more) anatomical features F1, F2. The output provided by the system may allow a user to determine whether there is an interface represented by the image, and preferably, where the interface is travelling. Thus, the output may only provide a binary indication, such as an acoustic signal, or a visual signal or any other sensory output which merely informs the user that such a sliding interface exists. Preferably the output goes beyond this and provides an indication within an image, wherein the interface is presented in the input image.

[0048] For example, in one embodiment, the visualizer VIZ may use the intra-image indication to generate a graphical display GD, the graphical display comprising a graphical representation r(IF) of a sliding interface along which one or both of the anatomical features AF1, AF2 move. Depending on the dimensionality of the input image, two-dimensional (2D) or three-dimensional (3D) images are contemplated herein, respectively, and the representation r(IF) may be a linear structure or a 2D surface in a 3D space. The indication r(IF) of the sliding interface IF may be displayed in different colors or gray values, shades, tones or other visual modulations relative to the rendering of the surrounding image elements (voxels or pixels). In the following, the term "image element" may sometimes be used herein to refer generally to pixels or voxels, depending on the dimensionality of the input image. In addition to providing an indication of the position of the interface IF in the image by means of a graphic in the graphical display GD, the mentioned sensory signals (sound, visual, tactile, etc.) may also be provided as auxiliary warning signals. The output graphical representation r(IF) may be included as an overlay structure on one or both of the input images I1, I2 to indicate the position of the interface IF. The graphical representation r(IF) can outline the position and path of the interface. If necessary, the graphical indication can be displayed without the input image, but this is not preferred. In addition to the graphical representation, the position of the interface IF can also be provided in the form of a set of coordinates in pure digital form.

[0049] Indicating the position of the interface in the image based on at least one of the input images I1, I2 is preferred, as this allows a user (such as medical personnel) to quickly determine the position of the interface IF. This knowledge can provide information for further diagnostic or treatment steps. The position of the interface IF provided by the system SYS can be used to control other medical equipment, such as a surgical robot or any other treatment device, such as a radiation therapy device (LINAC) or any other medical equipment. In some embodiments, the indication can be provided as a control signal to control the operation of such or other medical equipment (such as a contrast pump, other or the same imaging device, etc.).

[0050] The output data of the system SYS encode the position of the interface IF and can be stored in the memory MEM together with or in association with one, some or all of the input images I1 , I2 .

[0051] Such (one or more) sliding interfaces IF may appear in various scenarios between anatomical features AF1 and AF2, such as imaging of lungs or imaging of joints (such as human knees or elbows). In the former case of lung imaging, such interfaces IF may appear in interlobar motion between lobes AF1, AF2, or in the motion of lobes relative to the pleura relative to the thoracic cavity, or others. In joint imaging, such an interface can be defined by the motion of two bone parts that meet at a joint, such as the motion of a ball to a socket in a spherical joint or a hinge joint. The sliding motion in both cases may be caused by breathing or joint actuation (such as knee extension, etc.). During the sliding motion, the two features AF1, AF2 do not need to be in direct contact. These features may be coupled by interstitial substances (such as fat, fluid or cartilage). Alternatively, one of the features AF1, AF2 may be related to such interstitial substances, while the other feature refers to a bone part or other tissue type that is coupled to the substance. In this article, both features do not have to move in the same or opposite directions to define a sliding interface (such as Figure 1 ), as it is sufficient that one of the features AF1 slides over the other feature AF2. The sliding interface IF is the trajectory (usually a 2D surface) where the sliding motion occurs. A specific amount of shear may be induced, depending on adhesion or other coupling or friction between the two features AF1, AF2. The interface may travel in a bulk of the same type of tissue. Therefore, two anatomical features may be sub-features of the same anatomical feature, and so do not necessarily refer to different materials or tissue types. For example, the interface may define a cleft or fracture in a bone, or may be defined as an interlobar fissure. However, in other cases, the two features may refer to different entities, made of different materials. Therefore, the sliding interface IF may or may not define a material interface. Clinical examples where knowing the location of the sliding interface is helpful may include image-based measurement of articular cartilage thickness, image-based lung function analysis, image-based alignment of respiratory-gated images, or to a lesser extent (but not exclusively) for radiation treatment planning.

[0052] Reference now Figure 2 , the system SYS may include a sliding motion evaluator SME, a sensitivity analyzer SA and a sliding interface detector SD as functional components. In operation, the input port IN of the system SYS receives two (or more) input images I1, I2, for example in the online or offline setting mentioned above. The sliding motion evaluator SME analyzes the input images and generates a spatial map M for the input images I1, I2. Specifically, a series of such maps M=M are generated for the input images I1, I2 jPreferably, the size of the map M corresponds to the size of the input image. Thus, some or each map has the same spatial structure in terms of image elements as any, both or all of the input images I1, I2. j Corresponding to the vector field DVF j And M j Each spatial entry of represents the amount of response to a motion at a certain image position i that occurred during the acquisition time of the input images I1, I2. Figure 2 As shown, M j Each entry of can be the corresponding vector field DVF associated with the corresponding image position i j Encoding, and this is done for a plurality (eg all) of image positions of the input images I1, I2. j Each spatial entry of represents such a vector field, and each entry can be computed as a function of such a vector field. This function may be referred to herein as the metric μ. The metric μ can be implemented using a metric operator O acting on the vector field. Thus, for a given mapping M in the series j , M j( i)=O(DVF j )(i). Therefore, in an embodiment, the graph M may also be referred to as a motion metric graph of the image processing algorithm used by the sliding motion evaluator SME.

[0053] In the context of some such image processing algorithms contemplated herein, one of I1, I2 may be designated as the reference image for which the vector field DVF is measured. j One may refer to image I1 as a reference image purely on conventional grounds, but it is to be understood that this is by no means limiting: any of the one or more other input images I2 may be designated as a reference image.

[0054] Series M j may be obtained by changing the value of a parameter of an algorithm implemented by the sliding motion evaluator SME by the sliding motion evaluator SME. Thus, the sliding motion evaluator SME may implement a parameterized image processing algorithm. The image processing is configured to respond to the motion. The sliding motion evaluator SME may include an iterative loop component LC which, for each new parameter value p j Re-run Figure M j The parameter p may be changed by the loop component LC in predefined intervals. The interval can be set as a system variable or the system includes a user interface UI for the user to change the interval. The step size can be set and can also be changed through the user interface UI. The change of the algorithm parameter p with the loop component of the sliding motion estimator SME is determined by Figure 2 The value p in j. The variation of parameter p may involve multiple instances. In some cases, two instances may be sufficient, but preferably, the variation is more than two. Three or more instances of p may be preferred. For example, three may be sufficient and may achieve a useful balance between performance and responsiveness. Four or five instances of p may also achieve a similar effect. More than five instances are contemplated in many implementations. For example, the number of instances may be tens or hundreds, or even more, if desired.

[0055] The sliding motion analyzer SA converts a series of graphs M j As input. Based on the input graph series, use the operator D to build the sensitivity graph M s , D(M j )=M S The sensitivity map preferably has the same dimensions as the reference map I1. s Configured to measure motion map M j How to vary locally through the series M as a function of p.

[0056] In one embodiment, the operator D used by the sliding motion analyzer SA is a difference operator D, such as a differential operator on p. The operator D can be configured to approximate the differential or gradient of the graph series M, so M s = dM / dp. For example, if the metric μ is used, then the graph dM / dp might represent how the metric value responds to motion on the image and to changes in the image processing algorithm parameter p.

[0057] Sensitivity map M s The analysis can be performed by a sliding motion detector SD, for example using thresholding or other methods. A spatial mask of the interface position IF can be constructed in this way. The threshold can be found by experimentation or machine learning. If necessary, the user can adjust the threshold through an appropriate interface. The sensitivity map can be remade for each threshold setting and preferably displayed dynamically, with or without the input image I1 or I2. This dynamic threshold adjustment option with visual inspection allows the user to find the best spatially resolved representation of the interface position.

[0058] The output of the detector SD can again be represented as a spatial mask M such as φ A mapping of which, for each element, indicates whether the corresponding image element indicates a part of the sliding interface SI. φ Can be stored for further processing. In the embodiment interface diagram M φ Can be passed to the visualizer VIZ. The visualizer VIZ will convert the interface image into φThe visualizer may use the spatial information so mapped to drive the video circuitry of the display device DD to enable display of the graphic display GD on the display device DD. As described above, the graphic display may include generating an image overlay based on the interface map to indicate the location where the interface IF is located in at least one of the input images I1, I2. φ It can be used to highlight the corresponding image portion in at least one of the input images I1, I2, which indicates the position and (optionally) the range of the sliding interface IF. In other embodiments, discrete indicator symbols (such as one or more crosshairs or arrows, etc.) are arranged to indicate the position of the sliding interface IF. φ Any other visualization schemes are contemplated in this paper.

[0059] Surprisingly, we found that one suitable type of image processing algorithm that can be used by the sliding motion estimator SME is a non-rigid type of image registration algorithm. For example, some such algorithms are formulated in terms of optimizing an objective function F, which uses regularization or smoothing terms.

[0060] More specifically, image registration can be formulated as iteratively minimizing the joint sum of so-called "internal" and "external" forces. Conceptually, the external forces come from the chosen similarity measure (e.g., sum of squared differences, (local) cross-correlation, mutual information, normalized gradient field) and are therefore based on image-based features (intensity, edges, etc.). On the other hand, the internal forces are linked to a regularization term (or "deformation model") that penalizes the spatial gradient of the displacement in a simpler setting or incorporates more complex terms, such as from physically driven elastic models.

[0061] Starting from an initial deformation (e.g., zero displacement), the combined sum of the internal and external forces decreases at each iteration step. Convergence is achieved if the internal and external forces are in equilibrium, i.e., when an equilibrium point is reached.

[0062] Since the cost function consists of two parts, internal and external, the optimization task is to negotiate between two (or more) criteria: i) the solution (T j ) relative to the similarity (external force) of the “input data” in images A and B and ii) the solution (T j ) confirms the desired properties (such as smoothness defined by the internal force component of the cost function, etc.).

[0063] The objective function to be minimized can be formulated based on the sum of (at least) two functions, where one function D corresponds to the similarity measure and the other function S corresponds to the regularization or smoothing term,

[0064] F = D[u]+S[u]→min (1)

[0065] where u ranges over the space of registration transformations T. u is a deformation vector field, with respect to which the transformation T can be described. This is because closed analytical representations for the "formula" u are usually not available. The functions D and S may contain further parameters, such as the mentioned image processing algorithm parameters p, for example, parameters controlling the amount of elasticity, or in other words, parameters controlling the local homogeneity or heterogeneity of the resulting deformation vector field.

[0066] For example, p can be configured as follows:

[0067] In the function D, p can be used to weight the influence of the similarity measure upward or downward, for example, p can be a weighting factor or an additional weighting factor. Choosing a value greater than 1 for p will result in a registration output in which the deformation vector field is less normalized.

[0068] In the function S, p can be used to control one or more parameters of the regularizer. The regularizer can act on a combination of spatial derivatives of the directional components of the function u. A simpler example is to define the regularizer term as the sum of squares of first-order spatial derivatives, such that the first component of u is derived with respect to the first direction, the second component of u is derived with respect to the second direction, and so on. More complex examples involve derivative terms where the k-th component of u is derived with respect to the second direction. 1 The components are relative to the direction k 2 and k 3 The parameter p can be used as a weighting factor for one or more of these terms, e.g.

[0069]

[0070] Choosing a value for p greater than 1 will result in a registration output where local changes in one component of the deformation vector field are more closely correlated with local changes in another component of the deformation vector field.

[0071] The above are non-limiting examples of how the parameter p is configured, how it works, and how it is used. In other settings, the parameter may be configured or embedded in the current registration algorithm in a different way.

[0072] The output of interest in this registration algorithm (1) is the vector field u = DVF i , as described above. The vector field may in particular be of the type of deformation vector field, which describes how regions or volume elements need to be deformed in the registration.

[0073] As previously described, the metric μ may be defined by the evaluator SME based on one, two or more such registration algorithm outputs per image position i. In particular, one or more deformations may be provided as outputs per image position i. The deformation vector field may be analyzed by the analyzer SA according to the smoothness parameter p (or more generally the regularization parameter p) of the registration algorithm. It has been found that regularized non-rigid image registration type algorithms provide a good response to sliding motions. This response may be appropriately amplified by selecting the metric μ. Such types of amplified metrics μ may be used in conjunction with the registration algorithm or other algorithms, and will now be described in more detail.

[0074] Figure 3A -C shows Figure 2 , wherein a non-rigid type registration algorithm R is used. The output of the non-rigid type registration algorithm R, such as the deformation vector field DVF i , analyzed by analyzer SA.

[0075] First reference Figure 3A , setting various parameter values ​​p of the registration algorithm R j And when processing the input images I1 , I2 , a loop component of the evaluator SME loops over them. Figure 3A Different instances of the same registration algorithm R are shown, one for each parameter value in the loop, to allow for parallel processing. However, sequential processing is also envisaged, in which case a single instance R is sufficient for processing, followed by j .

[0076] The sliding motion evaluator SME may implement a registration algorithm R. The output of the registration algorithm R (eg, a deformation vector field) is processed for some or each image position i of the reference image I1 by an evaluator component EV of the sliding motion evaluator SME to generate an image M j series, for example by varying the parameter p. j The entries in may represent the calculations by applying the operator O of the metric μ to the output provided by the registration algorithm R (such as the deformation vector field). j The output series is analyzed by a sensitivity analyzer SA, for example by generating a sensitivity map M s , whose entries represent the corresponding approximation of the local gradient dM / dP.

[0077] Figure 3B , 3C An embodiment of the evaluator component EV is shown in more detail. Figure 3AIn one embodiment, the estimator may include a spatial operator O with a single input per single vector field and image position. Thus, the operator O may compute the shear force per voxel, or the volume change per voxel per image position. Thus, the estimator may be based on a single deformation vector field DVF per image position i j Conduct an assessment.

[0078] exist Figure 3C In the multi-input processing embodiment shown on the right, the evaluator EV can use an operator O with at least two registration outputs per image position i as input for calculating the corresponding entries of each image. The at least two registration outputs are due to different parameter settings p, as will be explained in more detail below. The metric μ implemented by the operator O can be the vector field difference of two or more vector fields per vector position. Euclidean distance (squared or non-squared) can be used, for example, such as caused by the norm space Lp, p=2. Metrics caused by the norm Lp, other values ​​for p besides 2 are also envisioned, as well as any other suitable metrics, such as weighted metrics.

[0079] For example, the evaluator EV computes the graph series M j Other measures may include one of the following:

[0080] In one embodiment, the metric μ is based on the Jacobian determinant ("Jacobian term") of the deformation vector field. The Jacobian matrix responds to local volume changes. A Jacobian value of 1 corresponds to volume conservation. Therefore, two images acquired under different respiratory states will result in a Jacobian value close to 1 outside the lungs, and a Jacobian value inside the lungs that is generally not 1.

[0081] Choosing p to represent a relatively stiff elasticity results in a more uniform DVF. For example, the Jacobian at the pleura is affected by both the volume-preserving tissue outside the lung and the volume-changing tissue inside the lung.

[0082] On the other hand, choosing p to indicate a highly elastic setting results in a more inhomogeneous deformation vector field. For example, the Jacobian values ​​of lung voxels close to the pleura are less affected by tissue outside the lung. Therefore, the absolute difference dM(.) = |Jacobian(rigid) - Jacobian(elastic)| is highest in the transition region between volume-preserving tissue and volume-changing tissue, which corresponds exactly to the sought sliding interface IF.

[0083] In another embodiment, the metric μ can be selected based on the maximum shear stress of the deformation vector field ("maximum shear"). This metric can be obtained by the principal stretch calculation of the eigenvalue decomposition of the stretch tensor. Image voxels affected by the sliding motion will get the maximum shear as positive value.

[0084] Similar to the Jacobian matrix in the previous example, the choice of elasticity value affects the uniformity of the maximum shear map. While for relatively stiff elasticity values ​​p, only image regions with non-local (extensive) slip are detected, p values ​​representing settings with higher elasticity may allow detection of possible slip locations in a localized range including false positive findings.

[0085] Using the proposed system SYS in a registration setting according to the above formula (1), the parameter p is configured to control the regularization operator, and a careful balancing act may be required to reliably and robustly detect the sliding interface IF. Choosing a relatively low value for the parameter p will overestimate the local sliding, resulting in a large number of false positive detections. Choosing a relatively high value for the parameter p results in quite drastic deformations, with few or zero local sliding detections.

[0086] The proposed system allows p to vary and uses dM / dp to avoid having to determine the optimal range for p. The proposed system can fully automatically detect sliding interfaces without relying on the user to know a priori the useful working range to achieve the aforementioned balance between a large number of false positives on the one hand and zero results when no such interface is found on the other hand.

[0087] Reference now Figure 4 , which shows the steps of a computer-implemented method for image-based detection of a sliding interface associated with two or more anatomical features in a patient.

[0088] The input received in step S410 includes at least two input images I1, I2 of the patient acquired by a suitable imaging method when such sliding motion occurs relative to at least two anatomical features. For example, images may be acquired during a running cycle, during knee flexion or extension, or during other joint actuations, depending on the situation.

[0089] In step S420, the severity or other aspects of the sliding motion are evaluated. In some embodiments, this can be implemented by applying an algorithm based on input images (such as a registration algorithm), in which one input image is used as a reference image. The evaluation results in a spatial map, which contains entries that can respond to the sliding motion recorded in the two input images. The output of the registration algorithm can be mapped to a scalar value per image element of the reference image or the other of the input images using a metric. The output of the registration algorithm may include a vector field, such as a deformation vector field, with one or more such vector fields per image element. The metric is configured to respond to such sliding motion, in particular the image structure dynamics caused by such sliding motion. The metric may amplify the response of the registration algorithm. Such a metric can be calculated based on the output of the registration algorithm applied to the two input images. The parameters of the registration algorithm can be changed to calculate multiple maps in the evaluation. These maps are spatial data structures that represent the entries per image position, such as per image position of the reference image I1 or the input image I2.

[0090] In step S430, the series of maps are analyzed to generate a sensitivity map. This can be achieved by using a difference operator to calculate an approximation of the gradient of the metric map with respect to the parameter. The sensitivity map may have the same size as any one of the metric maps and / or any one of the at least two input images.

[0091] In step S440, the sensitivity map generated in step S430 is analyzed, for example by thresholding, to provide an indication of the location of the interface. Thus, the output of step S440 may be a map (such as a spatial mask) for one of the two input images, wherein each entry indicates whether the corresponding image location represents a portion of the sought sliding interface associated with the two anatomical structures.

[0092] In step S450, the output of step S440 may be further processed to provide an indication of the presence or location of an interface based on the input imagery. For example, an interface mask / map may be displayed on a display device. For example, a graphical representation of the map, appropriate color coding or visual modulation may be superimposed on the motion and superimposed on at least one of the two input images.

[0093] Any other processing is also contemplated herein to produce such an indication that is suitable for user action. For example, in one embodiment, such positioning may not be required, and it may be sufficient to provide a signal to indicate whether such an interface exists in the case recorded by two or more input images. In such a binary type indicator embodiment, only the induction of whether such an interface IF is sought, and the evaluation step may not produce a graph, but rather a scalar value applicable to the global input image. For example, a single vector field may be provided for each parameter p of the entire image I1 or I2, for example, an average vector field or any other quantity that responds to a sliding motion may be used.

[0094] In some embodiments, the above method can be implemented as follows.

[0095] p is a parameter of the registration algorithm, preferably a parameter controlling the amount of elasticity, or a parameter controlling the weight of the regularization term compared to the similarity term, or more generally, a parameter controlling the smoothness of the resulting deformation.

[0096] Let μ be a metric evaluated voxel-by-voxel on the output of the registration algorithm, e.g., the determinant of the Jacobian matrix of the deformation vector field or the difference between the maximum and minimum absolute eigenvalues ​​of the stretch tensor of the deformation vector field.

[0097] The parameter p can be initialized by setting the initial value v 0 is initialized, usually to achieve good alignment. The parameter p then loops through a set of values ​​{…, p -3 ,p -2 ,p -1 ,p 0 ,p 1 ,p 2 ,p 3 ,…}. For each such value p j , run the registration algorithm with the selected value for p. Registration result, deformation vector field DVF j , is stored in memory. Optionally, the metric μ is used to evaluate the output DVF j And store the metric output as a map M j as part of a series M. As mentioned before, since the registration result is per image position i, M j is a map of the same size as the input image, e.g. the size of the reference image. s can be calculated as the gradient of M with respect to p, dM / dp. s A good approximation to this is to use the difference operator to compute the difference sequence dM(p -2 ):=M(p -2 )-M(p -3 ),dM(p -1 ):=M(p-1 )-M(p -2 ),dM(p 0 ):=M(p 0 )-M(p -1 ), etc. The sensitivity map Ms can then be analyzed (e.g. by thresholding) to obtain the sliding interface localization map Mφ. For the case where the metric μ is implemented by an operator O as the vector field difference of two or more vector fields, it is preferred to analyze M instead of dM.

[0098] M(p) is usually the same size as the input image, so if I1 or I2 is 3D (image volume), then the gradient dM is a 4D volume (3D+t). If I1 or I2 is 2D, then the gradient dM is a 3D data structure (2D+1).

[0099] A zero gradient dM corresponds to μ being insensitive to the parameter p. This fact can be used to conclude that either μ is improperly chosen or p is improperly chosen. The choice may be improper because μ or p or both have no effect on the registration output. In addition, a zero gradient may indicate that the input images I1, I2 are (almost) identical or that the input images consist of constant intensity, etc. Therefore, in an embodiment, the system may include a sanity check loop SCL. The loop SCL analyzes the gradient dM, and if a zero value is found (optionally, within a predefined range), an alarm signal is issued, preferably by displaying any or more of the above options on a display device DD. Therefore, the sanity check loop SCL can support the user in exploratory work by finding a suitable parameter selection. SCL may support a user interface UI so that the user can change either or both of μ and p. After receiving a change request via the user interface UI, the gradient dM is dynamically re-evaluated for μ and p.

[0100] Changing a parameter p, such as the elasticity parameter in a non-rigid registration algorithm, may result in registration inaccuracies over a larger range. Both very low and very high elasticity values ​​may result in the registration algorithm being unable to align the main image structures of the image pair I1, I2. This can be avoided by choosing a suitable deformation vector field as initialization in the iterative registration algorithm. The following approach can be used: a first registration can be run using values ​​for p that are known to result in good overall alignment. Then, a second registration is run, using the output of the first registration as initialization for the deformation vector field, but now using values ​​for p chosen from a larger range.

[0101] With the proposed system and method, there is no longer a reliance on potentially unstable segmentation algorithms. Therefore, the proposed sliding interface detection preferably does not use image segmentation. In addition, the proposed method includes changing the registration parameter p to obtain a series of metric maps. This operation of changing the parameter solves the problem of not knowing p. However, there may still be extreme (low or high) choices for p that lead to inaccurate alignment. To overcome this, the above method is based on two registration runs: a first registration is run using a typical p, and then a second registration is performed using the output of the first registration as input, where p can be selected from a large range. It is worth noting that although the selection of a typical parameter p does not constitute a technical problem from the perspective of registration for the first registration (because a good alignment may be obtained by selecting from a large range), it may be problematic for sliding interface detection (which is the main focus of this article), and the above-mentioned proposed method provides a series of technical means to overcome this problem.

[0102] The proposed system SYS and method can be integrated as an optional feature in scanner console imaging such as CT, MR, US. In addition, the proposed system SYS and method can be integrated as an option into a workstation for image post-processing.

[0103] The components of SYS may be implemented as one or more software modules, running on one or more general purpose processing units CU, such as a workstation associated with an imager IA, or on a server computer associated with a group of imagers.

[0104] Alternatively, some or all components of the system SYS may be arranged in hardware, such as a suitably programmed microcontroller or microprocessor, for example an FPGA (field programmable gate array) or a hardwired IC chip, an application specific integrated circuit (ASIC), integrated into the imaging system IA. In yet another embodiment, the system SYS may be implemented partly in software and partly in hardware.

[0105] The different components of the system SYS may be implemented on a single data processing unit CU. Alternatively, some or more components are implemented on different processing units CU, possibly arranged remotely in a distributed architecture and connectable in a suitable communication network, such as in a cloud setup or a client-server setup or the like.

[0106] One or more features described herein may be configured or implemented as or utilizing circuits encoded in a computer readable medium and / or a combination thereof. The circuits may include discrete and / or integrated circuits, systems on a chip (SOCs) and combinations thereof, machines, computer systems, processors and memories, computer programs.

[0107] In a further exemplary embodiment of the present invention, a computer program or a computer program element is provided, characterized in that it is adapted to perform the method steps of the method according to one of the preceding examples on a suitable system.

[0108] The computer program element can therefore be stored on a computing unit, which can also be part of an embodiment of the present invention. The computing unit can be suitable for executing the steps of the above method or causing the execution of the steps of the above method. In addition, it can also be suitable for operating the components of the device described above. The computing unit can be suitable for automatically operating and / or executing the user's command. The computer program can be loaded into the working memory of a data processor. The data processor can therefore be equipped to implement the method of the present invention.

[0109] This exemplary embodiment of the invention covers both a computer program that right from the beginning uses the invention and a computer program that by means of an update turns an existing program into a program that uses the invention.

[0110] Furthermore, the computer program element may be able to provide all necessary steps to complete the procedure of the exemplary embodiment of the method as described above.

[0111] According to another exemplary embodiment of the present invention, a computer readable medium, such as a CD-ROM, is proposed, wherein the computer readable medium has a computer program element stored thereon, the computer program element being as described in the previous section.

[0112] The computer program may be stored and / or distributed on a suitable medium (particularly, but not necessarily, a non-transitory medium) provided together with other hardware or as part of other hardware, such as an optical storage medium or a solid-state medium, but may also be published in other forms, such as via the Internet or other wired or wireless telecommunications systems.

[0113] However, the computer program may also be offered over a network like the World Wide Web and can be downloaded from such a network into a working memory of a data processor. According to a further exemplary embodiment of the invention, a medium for making a computer program element available for downloading is provided, the computer program element being arranged to perform one of the previously described embodiments of the invention.

[0114] It must be noted that the embodiments of the present invention are described with reference to different subjects. In particular, some embodiments are described with reference to method-type claims, while other embodiments are described with reference to apparatus-type claims. However, as can be derived by those skilled in the art from the above and the following descriptions, unless otherwise indicated, any combination between features related to different subjects is also considered to be disclosed by this application, in addition to any combination of specific features belonging to the same type of subject. However, all features can be combined to provide a synergistic effect that is more than the simple sum of the said features.

[0115] Although the present invention has been illustrated and described in detail in the drawings and the foregoing description, such illustration and description shall be considered illustrative or exemplary, but not restrictive. The present invention is not limited to the disclosed embodiments. Other variations of the disclosed embodiments can be understood and achieved by those skilled in the art when practicing the claimed invention by studying the drawings, the disclosure, and the dependent claims.

[0116] In the claims, the word "comprising" does not exclude other elements or steps, and the words "a" or "an" do not exclude a plurality. A single processor or other unit can perform the functions of several items recited in the claims. Although specific measures are recited in mutually different dependent claims, this does not indicate that a combination of these measures cannot be used advantageously. Any reference signs in the claims should not be construed as limiting the scope. Such reference signs can consist of numbers, letters, or any alphanumeric combination.

Claims

1. An image processing system (SYS) for image-based sliding interface detection, include: an input port (IN) for receiving at least two medical input images (I1, I2), the at least two medical input images being acquired at different times for a patient (PAT) during a sliding movement of at least one of at least two anatomical features (AF1, AF2) of the patient relative to at least another of the at least two anatomical features (AF1, AF2), the sliding movement defining a sliding interface (SI) with respect to the at least two anatomical features (AF1, AF2); a sliding motion evaluator (SME) configured to apply a registration algorithm to the at least two images and to apply it with varying values ​​of parameters of the registration algorithm to compute a series of metric maps, the series of metric maps being configured to be responsive to the sliding motion registered in the at least two input images; a sensitivity analyzer (SA) configured to compute a sensitivity map representing changes in values ​​of said series of metric maps according to said changes in said parameter; as well as A sliding interface detector (SD) is configured to detect a position of a representation of the sliding interface in at least one of the at least two input images based on the sensitivity map.

2. The system according to claim 1, in, The parameter is a regularization parameter configured to influence a smoothing term based on which the algorithm is operable.

3. A system according to any one of the preceding claims, in, The registration algorithm is based on establishing a per-parameter vector field for at least one of the at least two images.

4. The system according to claim 3, in, The metric is based on the vector field.

5. The system according to claim 3 or 4, in, The vector field is a deformation vector field.

6. The system according to claim 5, in, The metric is based on any of: i) the difference between the vector fields, ii) the Jacobian matrix of the vector fields, iii) the eigenvalues ​​of the vector fields, (iv) an operator acting on the vector fields.

7. A system according to any of the preceding claims, comprising a visualizer (VIZ) configured to generate a graphical display (GD) for display on a display device (DD), the graphical display (GD) comprising an indication of the position of the representation of the sliding interface in at least one of the at least two images.

8. A system according to any one of the preceding claims, in, The detection operation performed by the sliding surface detector (SD) is based on threshold processing.

9. An arrangement (AR) comprising a system (SYS) according to any of the preceding claims and an imaging device (IA).

10. The arrangement according to claim 9, in, The imaging device (IA) is any one of the following: i) an imaging device configured for transmission imaging, ii) an imaging device configured for emission imaging, iii) a magnetic resonance imaging device, iv) an ultrasound imaging device.

11. A computer-implemented method for image-based sliding interface detection, include: receiving (S410) at least two medical input images (I1, I2), the at least two medical input images being acquired at different times for a patient (PAT) during a sliding motion of at least one of at least two anatomical features (AF1, AF2) of the patient relative to at least another of the at least two anatomical features (AF1, AF2), the sliding motion defining a sliding interface (SI) with respect to the at least two anatomical features (AF1, AF2); applying (S420) a registration algorithm to the at least two images, and applying it with varying values ​​of parameters of the registration algorithm, to calculate a series of metric maps, the series of metric maps being configured to respond to sliding motions recorded in the at least two input images; calculating (S430) a sensitivity map, the sensitivity map representing changes in values ​​of the series of metric maps according to the changes in the parameter; as well as A position of the representation of the sliding interface in at least one of the at least two input images is detected (S440) based on the sensitivity map.

12. A computer program element, which, when run by at least one computing unit (CU), is adapted to perform the method according to claim 11.

13. A computer-readable medium having a program element according to claim 12 stored thereon.

Images