Method and system for image processing - Patents.com

Abstract

According to one aspect, a computer-implemented method is provided for determining edits to be applied to a 3D mesh representing a division of a 3D image, the computer-implemented method comprising: in response to receiving an indication of an adjustment to be applied to a first location on the 3D mesh within an editing plane of the 3D image, adjusting a point on a boundary of the 3D mesh within an editing region of the 3D mesh using an anisotropic weighting factor that limits the adjustment made to the point if the point is in a first orientation relative to the editing plane.

Description

[Technical field]

[0001] The present invention relates to image processing, and more particularly to image processing for determining edits to be applied to a three-dimensional (3D) mesh. [Background technology]

[0002] The present disclosure is in the field of 3D mesh manipulation of 3D meshes generated as a result of segmentation of an image. The disclosure herein applies to a range of images, such as, for example, medical images. Image segmentation involves extracting shape / morphological information about objects or shapes captured within an image. This is accomplished by converting the image into constituent blocks or "segments" that represent different features of the image. Image segmentation involves fitting a model to one or more features of the image.

[0003] One method of image segmentation is model-based segmentation (MBS), whereby a triangulated three-dimensional (3D) mesh of an object structure (e.g., heart, brain, lungs, etc.) is iteratively adapted to image features. Segmentation models typically encode population-based appearance features and shape information. Such information represents the allowed shape variations based on the actual shapes of the object structures of the members of the population. Shape variations are encoded, for example, in the form of eigenmodes that represent the manner in which changes to one part of the model are constrained or dependent on the shapes of other parts of the model.

[0004] Model-based segmentation has been used in a variety of applications to segment one or more object organs from medical images, see, for example, Ecabert, O. et al., 2008, entitled "Automatic Model-Based Segmentation of the Heart in CT Images," IEEE Trans. Med. Imaging 27(9), 1189-1201. The use of triangulated surface meshes has generally resulted in MBS achieving smooth segmentation results. Furthermore, MBS is generally considered to be robust to image artifacts, such as variations in image quality.

[0005] Interactive 3D mesh editing is typically used to manually fine-tune automatic segmentation results, such as those described above. Once a model containing a 3D mesh is created, the 3D mesh is displayed on the image, e.g., as an overlay on the image that was used as the basis for segmentation. A user can edit the mesh by dragging the mesh region to be edited to a desired target location so that the 3D mesh better matches the object in the image represented by the 3D mesh. Mesh edits are propagated from the point where the mesh is edited to other points on the mesh, but are typically constrained to a certain percentage of the mesh from the point where the mesh is edited. This is referred to as the edit region or area of ​​influence. To allow efficient editing of 3D meshes, it is desirable to improve the manner in which the edit region is defined. Summary of the Invention [Problem to be solved by the invention]

[0006] 3D volumes acquired using MR (Magnetic Resonance) typically have anisotropic resolution due to the acquisition process. 3D volumes resulting from MR analysis often have one particular axis with a very different resolution than the other axes. MR images are acquired slice by slice, often with full temporal information acquired consecutively for each slice, which can result in inconsistencies between slices due to changes in the region of the subject corresponding to the slice that occur over time.

[0007] A 3D mesh is generated for a 3D image, such as an MR (Magnetic Resonance) image, made of multiple slices, using a segmentation method such as the one described above. The user then edits the 3D mesh so that the mesh better matches the object it represents. A problem with 3D editing tools in MR applications is that the user interface only allows per-slice editing of the 3D mesh. In cardiac MR, this means that often only short-axis slices are in the process of being edited. 3D mesh editing tools typically use an influence region that falls off isotropically along all spatial dimensions. A small edit region limits the edit to the slice, but does not allow larger edits within the slice. A large edit region can result in a large edit area within the slice, but also has undesirable effects on neighboring slices.

[0008] It has been determined by the inventors that an improved configuration of edit regions is one in which a large area within a slice is affected, but the impact on neighboring slices can be limited. It is an objective of the embodiments herein to provide an improved method for determining edits to be applied to a 3D mesh. [Means for solving the problem]

[0009] Thus, according to one aspect, there is provided a computer-implemented method for determining edits to be applied to a three-dimensional (3D) mesh representing a division of a 3D image, the computer-implemented method responsive to receiving an indication of an adjustment to be applied to a first location on the 3D mesh within an editing plane of the 3D image, adjusting the point using an anisotropic weighting factor that limits the adjustment made to the point on a boundary of the 3D mesh within an editing region of the 3D mesh if the point is in a first orientation relative to the editing plane. This method is applicable to images acquired using a variety of imaging methods, such as MR, that typically build up an image by obtaining slices of the image, or in ultrasound, where the edits are performed by modifying the 3D mesh by interacting with a reformatted surface of an ultrasound image.

[0010] It is assumed herein that any standard process is used in addition to the anisotropic weighting factor to determine the adjustments to be made to the points. For example, the anisotropic weighting factor is applied to edits determined for the points based on the extension of a first positional adjustment to the points on the boundary of the 3D mesh. The edits are determined using any conventional 3D mesh editing process.

[0011] The points are intersections of a 3D mesh. The 3D mesh defines a boundary. The intersections of the 3D mesh may define points on the boundary of the 3D mesh. The method is applied to a number of points on the boundary of the 3D mesh using anisotropic weighting factors that vary depending on the distance of a particular point from a face (e.g., an editing face). The anisotropic weighting factors limit adjustments made to the point in a first direction.

[0012] The present invention therefore relates to restricting the editing of a 3D mesh to a portion of the mesh that is in a first direction relative to an editing plane using an anisotropic weighting factor. This advantageously allows for directional restriction of the editing region. The anisotropic (e.g. directional) weighting factor restricts the editing in a particular direction, e.g. the first direction. The anisotropic weighting factor restricts the component of the editing in a particular direction, which is the first direction. In particular, the editing of the 3D mesh is restricted in a direction that is perpendicular to the editing plane. The editing plane is included in a slice of the 3D image and is a central plane of the slice. This allows, for example, that the 3D mesh in a slice that has not yet been manipulated by the user is edited, while editing of slices that have already been manipulated by the user is prevented (or significantly reduced). For example, the 3D image includes multiple slices to be edited in succession, and the first direction is opposite to the direction in which the slices are edited. In particular, the editing of slices adjacent to the slice that includes the editing plane is restricted such that a zero adjustment is applied to the central editing plane of the slice that is adjacent to the slice that includes the editing plane. Moreover, this allows for large regions within the slice to be edited. The method is beneficial for images with anisotropic resolution (e.g., cardiac MR or prostate MR) since these images have high resolution in one direction but low resolution in another direction. Thus, editing is limited in low resolution directions, but less limited in higher resolution directions. For example, the resolution of a 3D image is high in directions parallel to the plane of the slice and low in directions perpendicular to the slice. Editing of a 3D mesh is limited in directions parallel to the axis of an anatomical feature of the 3D image. Limiting editing in a particular direction (such as perpendicular to the slice) allows for a smooth transition between the edited region and the frozen region (e.g., the region that should not be edited).

[0013] Editing of the 3D mesh is constrained in first and second directions perpendicular to the editing plane (e.g., directions that are opposite to each other and perpendicular to the editing plane). In particular, editing of the mesh is constrained in a first direction for points of the mesh that are in the first direction (e.g., perpendicular to the editing plane and above the editing plane) and editing of the mesh is constrained in a second direction for points of the mesh that are in the second direction (e.g., perpendicular to the editing plane and below the editing plane). Thus, editing of the mesh is constrained in the first and second directions perpendicular to the editing plane (e.g., so that the editing is not restricted within the slice or the mid-plane of an adjacent slice is not edited), but in the direction parallel to the editing plane, editing is constrained such that editing is not restricted by the anisotropic weighting factor. Thus, a large area within the slice is edited. The first direction may be, for example, a direction perpendicular to the editing plane, a direction parallel to an axis of interest, a direction parallel to an axis of an anatomical feature in the 3D image, a direction parallel to an axis of a ventricle in the 3D image, or a direction parallel to an axis of a prostate in the 3D image.

[0014] An anisotropic weighting factor is determined for a point based on a constraint function. In particular, the anisotropic weighting factor depends on the distance of the point from a plane (e.g., an editing plane). An edit region is a region where an adjustment applied to a first location on the 3D mesh results in an edit of other points on the 3D mesh that are within the edit region due to the extension of the edit to the other points. The edit region is defined by the adjustment applied to the first location. The edit region is a predefined distance from the indicated adjustment to be applied to the first location. The edit region is initially defined by an isotropic weighting factor. A constrained edit region is defined by a constraint function (e.g., the constraint function constrains the edit of points within the edit region). A constrained edit region represents a region where the edit of a point is not constrained to zero by an anisotropic weighting factor. A constrained edit region of edits decays quickly along a given direction (e.g., a first direction). In other directions (e.g., parallel to the editing plane), the edit region decays less quickly. To provide a consistent user experience for varying slice thicknesses, the width of the border or boundary of the restricted edit region along the slice normal and the edge sharpness (or smoothness) are specified relative to the slice thickness. Furthermore, the direction in which the edits are restricted may be only one direction or more than one direction. Different anisotropy weighting factors are applied in different directions or the same anisotropy weighting factor is applied in more than one direction. For example, different parameters related to the smoothness and the central location of the boundary of the restricted edit region are used to define different anisotropy weighting factors for different directions. The direction in which the edits should be restricted is input by the user. The anisotropy weighting factor is derived from a smooth weighting function and / or a sigmoid function.

[0015] According to a further aspect, there is provided a system for determining edits to be applied to a three-dimensional (3D) mesh, the 3D mesh representing a division of a 3D image, the system comprising: a memory having instruction data representing a set of instructions; and a processor configured to communicate with the memory and execute the set of instructions, the set of instructions which, when executed by the processor, cause the processor to, in response to receiving an indication of an adjustment to be applied to a first location on the 3D mesh within an editing plane of the 3D image, adjust the point using an anisotropic weighting factor that limits the adjustment made to the point if the point on a boundary of the 3D mesh within an editing region of the 3D mesh is in a first orientation relative to the editing plane.

[0016] According to a further aspect, there is provided a computer program product comprising a computer readable medium having computer readable code embodied therein, the computer readable code being configured, when executed by a suitable computer or processor, to cause the computer or processor to perform any of the methods described herein.

[0017] These and other aspects will be apparent from and elucidated with reference to the embodiments described hereinafter.

[0018] Example embodiments will now be described, by way of example only, with reference to the following drawings, in which: [Brief description of the drawings]

[0019] [Figure 1] FIG. 1 illustrates an apparatus for use with the methods described herein, according to an example. [Diagram 2] FIG. 1 illustrates a method according to an example. [Diagram 3] FIG. 13 is a diagram showing an example of an editing surface for a 3D image. [Figure 4]FIG. 2D depiction of an example stack of slices of a 3D image with a 2D depiction of a 3D mesh overlaid on the image. [Diagram 5] FIG. 1 shows an example of a 2D representation of a 3D image containing multiple slices. [Figure 6] FIG. 6(a) shows two graphs of the limiting function, where the effect of the value of the center of the boundary of the edit region μ is shown, and FIG. 6(b) shows the effect of the value of the smoothness parameter σ. [Figure 7] FIG. 7(a) shows single-sided and two-sided constraint functions, and FIG. 7(b) shows two graphs of constraint functions, showing the effect of changing the values ​​of the center of the edit area boundary and the smoothness parameter on the two-sided constraint function. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] As briefly explained above, editing regions of a 3D mesh are typically isotropic; however, an editing region may be either too large and affect adjacent slices, or too small and not affect a slice sufficiently.

[0021] It is an objective of the embodiments herein to provide a method and system for determining edits to be applied to a 3D mesh, which has been obtained by segmenting a 3D image acquired using a conventional imaging system, such as an MRI or ultrasound system.

[0022] Reference is now made to Figure 1. In some embodiments, there is an apparatus 100 for use in determining edits to be applied to a three-dimensional (3D) mesh, where the 3D mesh represents a division of a 3D image, according to some embodiments herein. Typically, the apparatus forms part of a computer apparatus or system, such as a laptop, desktop computer, or other computing device. In some embodiments, the apparatus 100 forms part of a distributed computing arrangement or cloud.

[0023] The apparatus comprises a memory 104 containing instruction data representing a set of instructions, and a processor 102 (e.g., processing circuitry or logic) in communication with the memory and configured to execute the set of instructions. In general, the set of instructions, when executed by the processor, causes the processor to perform any of the embodiments of a method for determining edits to be applied to a three-dimensional (3D) mesh as described below.

[0024] An embodiment of the apparatus 100 is for use in a system for determining edits to be applied to a three-dimensional (3D) mesh. More specifically, the set of instructions, when executed by a processor, cause the processor, in response to receiving an indication of an adjustment to be applied to a first location on the 3D mesh within an editing plane of a 3D image, to adjust a point on a boundary of the 3D mesh within an editing region of the 3D mesh using an anisotropic weighting factor that limits the adjustment made to the point if the point is in a first orientation relative to the editing plane.

[0025] The processor 102 may comprise one or more processors, processing units, multi-core processors, or modules configured or programmed to control the device 100 in the manner described herein. In certain implementations, the processor 102 may comprise multiple software and / or hardware modules, each configured or directed to perform an individual step or steps of the methods described herein. The processor 102 may comprise one or more processors, processing units, multi-core processors, and / or modules configured or programmed to control the device 100 in the manner described herein. In some implementations, for example, the processor 102 comprises multiple (e.g., interoperable) processors, processing units, multi-core processors, and / or modules configured for distributed processing. It will be understood by those skilled in the art that such processors, processing units, multi-core processors, and / or modules may be located in different locations and perform different steps of the methods described herein and / or different portions of a single step.

[0026] The memory 104 is configured to store program code that can be executed by the processor 102 to perform the methods described herein. Alternatively or additionally, the one or more memories 104 are external to (i.e., separate from or remote from) the device 100. For example, the one or more memories 104 are part of another device. The memory 104 can be used to store, for example, a 3D mesh, intersections and / or points on the boundary of the 3D mesh, a compilation of points on the boundary of the 3D mesh, an image, e.g., a 3D image, an edited surface of a 3D image, multiple slices of a 3D image, data related to the image, data related to anisotropic weighting factors, a limiting function, weighting factors, and / or any other information or data received, calculated, or determined by the processor 102 of the device 100 or from any interface, memory, or device external to the device 100. The processor 102 is configured to control the memory 104 to store a 3D mesh, intersections and / or points on the boundary of the 3D mesh, an edit of the points on the boundary of the 3D mesh, an image, e.g. a 3D image, an edit surface of the 3D image, a number of slices of the 3D image, data related to the image, data related to the anisotropic weighting factors, a limiting function, and weighting factors.

[0027] In some embodiments, memory 104 includes multiple sub-memories, each capable of storing a portion of instruction data, such as at least one sub-memory storing instruction data representing at least one instruction of the set of instructions, while at least one other sub-memory stores instruction data representing at least one other instruction of the set of instructions.

[0028] 1 shows only the components required to illustrate this aspect of the disclosure, it will be understood that in an actual implementation, the device 100 will include additional components to those shown. For example, the device 100 further includes a display. The display includes, for example, a computer screen and / or a screen on a mobile phone or tablet. The device further includes a user input device or interface, such as a keyboard, mouse, or other input device that allows a user to interact with the device, for example, to allow the user to provide an indication of an adjustment to be used in the methods described herein. The device 100 includes a battery or other source of power for the device 100, or a means for connecting the device 100 to a mains power source.

[0029] 2, there is a computer-implemented method 200 for use in determining edits to be applied to a three-dimensional (3D) mesh, where the 3D mesh represents a division of a 3D image. An implementation of method 200 is performed by an apparatus such as, for example, apparatus 100 described above.

[0030] Briefly, the method 200 includes, in response to receiving an indication of an adjustment to be applied to a first location on a 3D mesh within an editing plane of a 3D image 208, adjusting a point on a boundary of the 3D mesh within an editing region of the 3D mesh using an anisotropic weighting factor that limits the adjustment made to the point if the point is in a first orientation relative to the editing plane 210.

[0031] The received indication is based on a user input. The received indication is, for example, an indication of a movement of the 3D mesh from a first position to a second position on the 3D mesh to better match the object represented by the 3D mesh. The received indication of the adjustment is, for example, expanded to other points in the 3D mesh within the edit region. An anisotropic weighting factor is applied to the expanded edit to limit the expanded edit or to limit the adjustment made to points on the boundary of the 3D mesh in a particular direction.

[0032] The anisotropic weighting factor is determined for the point based on a constraint function. The constraint function is a smooth weighting function and / or a sigmoid function. The parameters of the constraint function include at least one of the distance of the point from the editing surface in the first direction, a smoothness parameter, and the first direction. Thus, the anisotropic weighting factor varies depending on the distance of the point from the editing surface in the first direction. For example, if a point on the boundary of the 3D mesh is at a large distance from the editing surface in the first direction, the constraint function tends to zero. If a point on the boundary of the 3D mesh is very close to the editing surface in the first direction, the constraint function tends to one. The adjustment of the point depends on the received indication of the adjustment, where the anisotropic weighting factor is applied to the adjustment extended to the first point from the indication of the adjustment at the first position.

[0033] It will be appreciated that the isotropic weighting factor is also applied to the edits of the mesh corresponding to an edit region around a first location on the 3D mesh edited by the user. In addition to the isotropic weighting factor, an anisotropic weighting factor is applied to the edits of the point to further constrain the edits of the point in the first direction. The constraining function constrains the edit region to define a constrained edit region. The constrained edit region is constrained relative to the edit region in at least one of the first direction and a second direction opposite the first direction.

[0034] The method is not limited to adjusting a single point on the boundary of the 3D mesh, but is applied to multiple points on the boundary of the 3D mesh, where any of the multiple points in a first direction relative to the editing plane (e.g., any point above the editing plane) are adjusted using an anisotropic weighting factor. The anisotropic weighting factor applied varies depending on the distance of the point from the plane (e.g., the editing plane). For example, a constraint function takes as input the distance of the point from the plane to determine the anisotropic weighting factor to be applied. The method is applied to an intersection that is a point on the boundary of the 3D mesh and / or to multiple intersections that are points on the boundary of the 3D mesh. An anisotropic weighting factor is determined for each of the intersections using the constraint function and based on the distance of the intersection from the editing plane in the first direction. Thus, the anisotropic weighting factor varies for each intersection.

[0035] Figure 3 shows an editing plane 312 of a 3D image, according to an example. In particular, Figure 3 shows an editing plane contained in a slice of a 3D image of a heart. Segmentation has been applied to the 3D image to create a 3D mesh of the heart. A 2D representation of the 3D mesh is shown in this figure as a dotted line 314. The user interacts with the 3D mesh by clicking and dragging directly on the image (in this example, clicking and dragging on the 2D representation of the 3D mesh 314).

[0036] Typically, to edit a 3D mesh, a user first looks at the image with the overlaid segmentation results to identify the area to edit. The user then clicks in the center of the area to edit and holds the mouse button down. The user then drags the mouse to the desired location while the displayed mesh is dynamically deformed (i.e., the area around the starting position is shifted towards the new mouse position). The mesh is deformed in 3D space. The user then releases the mouse button at the desired location. The deformed mesh is displayed and used for further analysis, such as volume calculation.

[0037] As discussed above, the deformation of the 3D mesh is initially confined to an edit area defined as a distance from the center of the edit area (e.g., the first point the user clicked), although typically the edits at the location edited by the user are propagated to other points within the edit area. The propagation of the edits falls off isotropically. Thus, a weighting is applied to the edits of nodes or points on the boundary of the 3D mesh within the edit area based on the distance of the point from the center of the edit area. The weighting is between 0 and 1, although the farther the point is from the edited point, the closer the weighting approaches a value of 0.

[0038] According to the examples herein, anisotropic weighting coefficients k between 0 and 1 for points on the boundary of the 3D mesh (e.g., each intersection of the 3D mesh) that determine what fraction of the edits used otherwise should be applied to the edits of the points on the boundary of the 3D mesh in order to limit the edits in a particular direction (which may be the first direction) rest is determined. The component of a point's edit that is parallel to the editing plane has a weighting of 1, while the component of a point's edit that is perpendicular to the editing plane has a weighting between 0 and 1. The anisotropic weighting factor is applied to the points within the editing region.

[0039] 4(a)-(c) show a stack of multiple slices of a 3D image 416 viewed as a 2D plane through the stack. In this example, the images show the subject's heart, although the method is applicable to any portion of the subject. The segmentation was applied to produce a 3D mesh that represents the structure of the heart. In the images, the 3D mesh is shown as a 2D representation 418. In each of FIGS. 4(a)-(c), a user has edited the mesh in an editing plane of an editing slice 420 (the editing is performed in a 2D image of the editing plane, such as that shown in FIG. 3). FIG. 4(a) shows an exemplary editing region 422 resulting from a user's edits, shown as a circular region. It will be understood that the dotted line in this image represents the border of the editing region, and further that the line is merely indicative of the border, and that the editing region has a smooth border that fades away. Points on the boundary of the 3D mesh that are within this region will be edited in response to the deformation of the 3D mesh. As shown in this figure, a typical edit region overlaps adjacent slices above and below the slice containing the edit plane 420. In particular, the edit region overlaps the central planes (e.g., edit planes) of each of two adjacent slices 424, 426. This is undesirable because if one or both of the adjacent slices have been previously edited by a user, further editing of the adjacent slices caused by editing the current slice would require the user to re-edit the previously edited adjacent slices.

[0040] FIG. 4(b) shows an example in which anisotropic weighting coefficients (or constraint functions) are used according to some examples herein. As shown in FIG. 4(b), the edit region 422 is constrained by the constraint function in a direction perpendicular to the edit plane 420, in two directions (first direction and second direction) both "upward" and "downward" relative to the edit plane. It will be understood that the direction perpendicular to the edit plane is not the only possible direction in which the edit may be constrained, and the direction in which the edit may be constrained may instead be a direction parallel to an axis of interest, such as the axis of the ventricle in the 3D image. The constraint direction may be selected anatomically, for example, based on the segmentation results. As an example, the edit may only be allowed above the mitral valve plane, and be constrained below. As shown in this figure, the edit of the 3D mesh is constrained within slices in the first and second directions perpendicular to the edit plane. Thus, the edit region does not overlap with the respective central planes of two adjacent slices 424, 426. However, in the direction parallel to the editing plane, the editing area 422 is not constrained (relative to the original editing area, or the editing area shown in FIG. 4(a)).

[0041] FIG. 4(c) shows a further example in which anisotropic weighting factors (or limiting functions) are used in accordance with some examples herein. In contrast to the example of FIG. 4(b), this example limits editing in only one direction perpendicular to the editing plane 420 (only the "up" direction). Thus, as shown in this figure, editing is limited within a slice in a first perpendicular direction relative to the editing plane. Thus, the editing region does not overlap the center plane of the first adjacent slice 424, but overlaps the center plane of the second adjacent slice 426. This is desirable when slices in one direction (e.g., upward) relative to the editing plane have been previously edited, but not desirable for slices in another direction (e.g., downward) relative to the editing plane. Typically, when reviewing the editing planes of slices and making edits to the 3D mesh, the user will work through a stack of slices (usually from top to bottom). Thus, using the methods described herein, the user does not need to revisit the previously edited surface, since it will not change based on further modifications of the subsequent slice made by the user. However, this particular configuration allows for a larger editing area in the plane of the slice and in the area extending below the editing surface, which results in a more rapid alignment of the 3D mesh with its actual position as indicated by the user.

[0042] It is noted that in this example, the edit region is constrained within a slice. However, it will be appreciated that the edit region is constrained such that editing of an adjacent slice occurs only up to the center face of the adjacent slice, where the edit is zero. In this manner, the center face (e.g., the previously edited edit face) of the adjacent slice is not altered based on the modification of the edit slice, but a smooth transition of the 3D mesh occurs between the edit faces of the slices.

[0043] When a 3D image is composed of multiple slices, as shown in Figure 4, the anisotropic weighting coefficients are determined as described below. In the examples below, the locations of points on the boundary of a 3D mesh are described as intersections on the 3D mesh. However, it will be understood that the invention is not limited to intersections and the method applies to any point on a 3D mesh.

[0044] To describe the position of the intersection point p along the slice normal direction z, relative to the current editing plane, a normalized offset from the current editing plane is determined using Equation 1 as follows: z=(pp plane )*n slice / Δ slice (1) Here, p plane is an arbitrary point in the editing plane (typically the center plane of a slice), and n slice is the slice normal, and Δ slice is the offset between the two slice centers.

[0045] Figure 5 shows a 2D representation of a 3D image comprising a number of slices 516, such as those described in relation to Figure 4. An edit slice 517 is indicated by a rectangle drawn with dotted lines and includes an edit plane 520, drawn as a dotted line. The edit plane is the central plane of the edit slice. The axis "Z" in Figure 5 represents the distance from the center of the edit slice and corresponds to the thickness of the slice, or the offset Δ between two slice centers. slice Each slice has a practical thickness Δ slice has.

[0046] The plane offsets are normalized (e.g., the offset is multiplied by the slice thickness Δ slice The editing plane is set with 0 offset (z=0). For point p, z is calculated using Equation 1. For p shown in Figure 5, z~1.2.

[0047] For points in the editing plane, z is 0. Points at an offset distance of one slice thickness along the slice normal have z=1, and in the opposite direction along the normal, z=-1. Normalizing the slice offset to the slice thickness advantageously allows geometric parameters to be defined independent of the slice thickness. Normalizing the slice offset can allow mesh editing properties to depend on the resolution and orientation of the image coordinate system.

[0048] Parameters that account for the width of the boundary of the constrained edit region (where the anisotropic weighting coefficients limit the adjustments made to the intersection, but not to zero) and the smoothness of the transition between 1 and 0 at the boundary are included in determining the edit to be applied to the intersection. These parameters are adjusted based on the current slice offset, not normalized. Alternatively, the width and sharpness (e.g., smoothness) of the border of the constrained edit region in the first direction is specified relative to the thickness of the slice that contains the edit plane. With normalized plane offsets, the parameters are selected at a more abstract level, such that a fixed set of parameters produces comparable results for different slice offsets.

[0049] Anisotropic weighting factor k rest To describe the decay of , a sigmoid function u is used that varies from 1 to 0, making it possible to choose the midpoint of the boundary of the edit region and the smoothness of the transition within the boundary (see Equation 2 below).

number

[0050] FIG. 6 illustrates, at 6(a), a sigmoid function as a function of z, where the effect of the center value of the boundary of the editing region μ on the sigmoid function is shown. In this example, values ​​of −0.2, −0.1, 0.0, 0.1, and 0.2 (corresponding to reference numbers 632, 634, 636, 638, and 640, respectively) are set as values ​​of μ (where lines for u(z) with these values ​​of μ are shown in this figure), and the value of σ is set as a constant value. As shown in this figure, by changing the center value of the boundary of the editing region, the boundary of the editing region is translated in the z direction relative to z=0. Thus, the value of μ can be selected based on the desired size of the editing region, where an increase in the center value of the boundary increases the size of the editing region (the weighting applied is non-zero) and therefore increases the distance from the editing surface that will give the point a non-zero weighting. For example, if a point is farther from the edit plane than the central plane of an adjacent slice (e.g., a distance of 1 from the edit plane), then a weighting of zero is preferably applied to the point, but if the edit is to be applied differently, a higher value of μ is used. Generally, values ​​of μ greater than zero are used. For very small values ​​that essentially give a weighting of zero (e.g., below a predefined threshold), a weighting of zero is applied.

[0051] FIG. 6(b) shows the sigmoid function as a function of z, where the effect of the smoothness parameter σ on the sigmoid function is shown. In this example, values ​​of 0.05, 0.07, 0.09, 0.11, and 0.13 (corresponding to reference numbers 642, 644, 646, 648, and 650, respectively) are set as the values ​​of σ, and the value of μ is set as a value of zero. As can be seen from this figure, the smaller the smoothness parameter σ, the steeper the transition from 1 to 0 around the central point (e.g., at the border of the edit region). Thus, the smoothness parameter can be varied to change the abruptness of the transition between a weight of 1 and a weight of 0. Typically, u(μ)=0.5 is used, where μ is a normalization offset where edits are applied with a weight of 50%.

[0052] Thus, the sigmoid function above can be considered to be a constraining function, except that to constrain editing in a first direction, e.g., upwards (e.g., in positive z) on one side of the current slice, the constraining function is constrained by an anisotropic weighting factor k rest Therefore, the anisotropic weighting factor k rest teeth, k rest,up =u(z) (3) It is defined as follows.

[0053] The values ​​of μ and σ are chosen so that the function value at z=0 is close to 1. Since z=0 corresponds to an edit slice, it is beneficial for the weighting to minimally restrict edits to points within the edit slice.

[0054] As an example, to restrict editing in a second direction, e.g., down the other side of the slice (e.g., negative z), the opposite direction of the first direction can be defined as -z, and thus the equation of u(-z) is used. therefore, k rest,down =u(-z) (4) get.

[0055] Thus, anisotropic weighting factors are applied in a first direction (e.g., above the slice) or in a second direction (e.g., below the slice), and it will be appreciated that μ and / or σ may be different for different directions (e.g., different for the first and second directions).

[0056] To constrain edits in both the first and second directions, the following equation is used: k rest =k rest,up *k rest,down (5)

[0057] Thus, a two sided constraint function is provided that allows for anisotropic weighting factors to be applied to points that are in a first direction relative to the editing plane, and for anisotropic weighting factors to be applied to points that are in a second direction relative to the editing plane.

[0058] Figure 7 shows single-sided and two-sided slice constraint functions. In particular, Figure 7(a) shows u(z) 726 as the constraint function in the first direction (in this example, it constrains edits above z=0 and has a weight of 1 below z=0) and u(-z) 728 as the constraint function in the second direction (in this example, it constrains edits below z=0 and has a weight of 1 above z=0). The figure also shows a constraint function u(z)*u(-z) 730 that constrains edits in both the first and second directions. Both single-sided functions are multiplied together to form the two-sided constraint function (solid line).

[0059] FIG. 7(b) shows examples of two-sided limiting functions for different parameter combinations (μ, σ). In particular, this figure shows limiting functions for parameter combinations (0.4, 0.09), (0.6, 0.07), (0.06, 0.09), (0.6, 0.11), and (0.8, 0.09) (corresponding to reference numbers 752, 754, 756, 758, and 760, respectively). As can be seen in this figure, the abruptness of the edit region boundary transition and the edit region size / edit region boundary are selected using different values ​​of μ and σ, where, as explained above, increasing the value of μ increases the edit region size and increasing the value of σ decreases the abruptness of the edit region boundary transition. Thus, the values ​​of these parameters are selected based on the requirements of the system. Although this figure shows two sided symmetric limiting functions, it will be appreciated that μ and / or σ are different in different directions (e.g., different for the first and second directions) and thus the limiting functions are asymmetric.

[0060] Note that in these examples the boundaries of the editing region are constrained such that the weighting coefficients reach a value of 0 at the mid-planes of adjacent slices (assuming the slices are of the same width). This means that the adjacent slices in the direction in which the anisotropic weighting coefficients are acting are not changed within the editing plane, so the user does not need to revisit previously edited slices, but a smooth transition for the 3D mesh is made between adjacent editing planes.

[0061] In another embodiment, a computer program product is provided that includes a computer readable medium having computer readable code embodied therein, the computer readable code being configured, when executed by a suitable computer or processor, to cause the suitable computer or processor to perform one or more of the methods described herein.

[0062] It will therefore be understood that the present disclosure also applies to computer programs, in particular to computer programs on or in a carrier adapted to put the embodiments into practice, in the form of object code, such as source code, object code, code intermediate sources and partially compiled forms, or in any other form suitable for use in the implementation of the methods according to the embodiments described herein.

[0063] It will be appreciated that such programs have many different architectural designs. For example, the program code implementing the functions of the method or system is subdivided into one or more subroutines. Many different ways of distributing functionality among these subroutines will be apparent to those skilled in the art. The subroutines are stored together in one executable file to form a self-contained program. Such an executable file includes computer executable instructions, such as processor instructions and / or interpreter instructions (e.g., Java interpreter instructions). Alternatively, one or more or all of the subroutines are stored in at least one external library file and linked with the main program, either statically or dynamically, e.g., at run-time. The main program includes at least one call to at least one of the subroutines. The subroutines further include function calls to each other.

[0064] A carrier for a computer program is any entity or device on which the program can be carried. For example, the carrier includes a data storage such as a ROM, for example a CD-ROM or a semiconductor ROM, or a magnetic recording medium, for example a hard disk. Furthermore, the carrier is a transmissible carrier such as an electric or optical signal, conveyed via an electric or optical cable or by radio or other means. When the program is embodied in such a signal, the carrier is constituted by such a cable or other device or means. Alternatively, the carrier is an integrated circuit in which the program is embodied and which is adapted to perform or used in the performance of the relevant method.

[0065] Variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the principles and techniques described herein, from a study of the drawings, the disclosure, and the appended claims. In the claims, the word "comprises" does not exclude other elements or steps, and singular elements do not exclude a plurality. A single processor or other unit performs the functions of several items recited in the claims. The mere recitation of certain methods in mutually different dependent claims does not indicate that a combination of these methods cannot be used to advantage. The computer program is stored or distributed on a suitable medium, such as an optical storage medium or a solid-state medium, provided together with or as part of other hardware, but also distributed in other forms, such as via the Internet or other wired or wireless communication systems. Any reference signs in the claims should not be interpreted as limiting the scope.

Claims

1. A computer-implemented method for determining an edit to be applied to a 3D mesh representing a segmentation of a 3D image, the method comprising: in response to receiving an indication of an adjustment to be applied to a first position on the 3D mesh within an edit plane of the 3D image; using an anisotropic weighting factor that restricts the adjustment to be made to a point when the point on the boundary of the 3D mesh within the edit region of the 3D mesh is in a first direction relative to the edit plane, the method comprising adjusting the point.

2. The method of claim 1, wherein the edit plane is at least one of included in a slice of the 3D image or a central plane of a slice of the 3D image.

3. The method of claim 1 or 2, wherein the first direction includes at least one of a direction perpendicular to the edit plane, a direction parallel to an axis of interest, a direction parallel to an axis of an anatomical feature of the 3D image, a direction parallel to an axis of a ventricle of the 3D image, and a direction parallel to an axis of a prostate of the 3D image.

4. The method of claim 1, wherein the anisotropic weighting factor for the point is determined based on a limiting function.

5. The method of claim 4, wherein the parameters of the limiting function include at least one of the distance of the point from the edit plane in the first direction, a smoothness parameter, and the first direction.

6. The method of claim 5, wherein the distance of the point from the edit plane in the first direction is normalized with respect to the thickness of the slice including the edit plane.

7. The method according to any one of claims 4 to 6, wherein the limiting function is at least one of a smooth weighting function and a sigmoid function.

8. The method of claim 4, wherein the limiting function restricts the edit region to define a restricted edit region.

9. The method of claim 8, wherein the restricted edit region is restricted with respect to at least one of the first direction and the edit region of at least one of the first direction and a second direction opposite to the first direction.

10. The method of claim 8 or 9, wherein at least one of the width and sharpness of the border of the restricted edit region in the first direction is defined with respect to the thickness of the slice including the edit plane.

11. The method according to claim 1, wherein when at least one of the following conditions is met: the point is outside the slice including the editing plane, or the point is in or beyond the editing plane of an adjacent slice, the anisotropic weighting coefficient restricts the adjustment made to the point to zero adjustment.

12. The method according to claim 1, wherein the 3D image includes a plurality of slices to be edited successively, and the first direction is opposite to the direction in which the slices are edited.

13. The method according to claim 1, comprising determining an editing region based on the received indication of the adjustment to be applied to the first position on the 3D mesh, wherein the editing region is the region where the boundary of the 3D mesh is to be adjusted.

14. A system for determining an edit to be applied to a 3D mesh representing a segmentation of a 3D image, the system comprising a memory containing instruction data representing a set of instructions, a processor configured to communicate with the memory and execute the set of instructions, and when the set of instructions is executed by the processor, the processor in response to receiving an indication of an adjustment to be applied to a first position on the 3D mesh within the editing plane of the 3D image, adjusts the point using an anisotropic weighting coefficient that restricts the adjustment made to the point when a point on the boundary of the 3D mesh within the editing region of the 3D mesh is in a first direction with respect to the editing plane.

15. A computer-readable medium incorporating computer-readable code configured to cause a computer or processor to execute the method according to claim 1 when executed by a suitable computer or processor.

Images