Method and system for evaluating 3D model of dental conditions

Abstract

A computer-implemented method for evaluating a digital 3D model of a tooth is disclosed. The method includes receiving a first digital 3D model representing a dental condition at a first time and receiving a second digital 3D model representing a dental condition at a second time. Further, the method includes generating an overlaid digital 3D model, and displaying a portion of the overlaid digital 3D model, the portion including the lesions of the dental condition identified on the first digital 3D model and the second digital 3D model. The displaying includes arranging a user-adjustable controller to vertically divide a displayed portion of the superimposed model into a first region and a second region. The displaying further includes identifying a first endpoint of the lesion in the first region and a second endpoint of the lesion in the second region; and arranging the first interactive window and the second interactive window.

Description

TECHNICAL FIELD

[0001] The present invention relates to a computer-implemented method and system for evaluating a digital three-dimensional (3D) model of a tooth. In particular, the present invention relates to a computer-implemented method and system for arranging a two-dimensional interactive window with respect to an overlaid digital three-dimensional (3D) model enabling observation of both the pathology of a dental condition and the interactive window. BACKGROUND

[0002] The development of intraoral scanning technology has played an important role in the transformation of modern digital dentistry. Using intraoral 3D scanners, dental practitioners can accurately and quickly capture a patient's dental situation, which can then be visualized on a display as a digital three-dimensional (3D) model display. Thus, the obtained digital 3D model can serve as a digital impression of the tooth, which has many advantages compared to traditional physical tooth impressions.

[0003] Scanning the dental situation of the same patient over a period of time can generate a series of digital 3D models, which can be compared to each other, for example, to track the development of dental conditions such as plaque, caries, gingivitis, gum recession, malocclusion, oral cancer, tooth wear, etc. Presenting the identified pathologies of dental conditions and annotations describing the pathologies on multiple digital 3D models can be challenging in a digital 3D environment. Known methods, including side-by-side views or sequential views of digital 3D models, do not provide sufficient flexibility for users to examine changes in dental conditions without relying too much on their personal skills. Furthermore, when interacting with a digital 3D model (e.g., when rotating a digital 3D model), a user can not see the annotations describing the identified pathologies and / or the pathologies themselves, as both the annotations and / or the pathologies can be hidden or misplaced during the interaction with the digital 3D model.

[0004] Another limitation of current solutions for annotating parts of a digital 3D model lies in the limitations in user interaction due to the difficulty of implementing interactive annotations in a digital 3D space. Currently available annotations, such as text boxes, drop-down menus, and input handler events, are well-established solutions in 2D space. Developing similar solutions in a digital 3D space would be beneficial.

[0005] There is a need to develop methods and systems to enable the continuous communication of clinically relevant annotations about how dental conditions are changing during the display of a digital 3D model and during user interaction with the displayed digital 3D model. Furthermore, it is desirable to enable users to interact with the annotations during the display of dental conditions on a digital 3D model. SUMMARY

[0006] The present disclosure explores how to leverage the capabilities of an intraoral 3D scanner system to enable a user to view identified lesions of dental conditions on multiple digital 3D models, inspect and interact with related annotations, and influence the diagnostic capabilities of the intraoral 3D scanner system based on the interaction.

[0007] In one embodiment, a computer-implemented method for evaluating digital 3D models of teeth is disclosed, the method comprising:

[0008] - receiving a first digital 3D model representing a dental situation at a first time;

[0009] - receiving a second digital 3D model representing a dental situation at a second time;

[0010] - generating a superimposed digital 3D model by aligning the first and second digital 3D models with each other;

[0011] - displaying a portion of the superimposed digital 3D model, the portion comprising a lesion of a dental condition identified on the first and second digital 3D models, wherein the display comprises:

[0012] - arranging a user-adjustable controller to vertically divide the displayed portion of the superimposed model into a first region and a second region, wherein in the first region only a portion of the first digital 3D model of the superimposed digital 3D model is rendered, and wherein in the second region only a portion of the second digital 3D model of the superimposed digital 3D model is rendered;

[0013] - identifying a first endpoint of the lesion in the first region and a second endpoint of the lesion in the second region;

[0014] - arranging a first interactive window at any point of the display device whose x-coordinate is less than or equal to the x-coordinate of the first endpoint of the lesion, the first interactive window comprising a severity value of the lesion identified on the first digital 3D model; and

[0015] - arranging a second interactive window at any point of the display device whose x-coordinate is greater than or equal to the x-coordinate of the second endpoint of the lesion, the second interactive window comprising a severity value of the lesion identified on the second digital 3D model.

[0016] The expression "3D" in the present disclosure refers to the term "three-dimensional". The term "digital 3D model" refers to a digital, three-dimensional, computer-generated representation of a patient's dental situation.

[0017] Such a digital 3D model can accurately correspond to the actual dental situation. This means that dental objects (e.g. teeth, tooth surfaces, restorations, and / or gingiva) on the digital 3D model can correspond to dental objects of the dental situation.

[0018] A digital 3D model can be constructed by the processor based on scan data collected during an intraoral scanning process, where an intraoral 3D scanner can be used to scan a patient’s dental situation, including teeth and gingiva. In this document, an intraoral 3D scanner is also referred to as an intraoral scanner. The digital 3D model can be stored in a memory of the computer system, e.g., in a standard triangulation language (STL) format or any other format for displaying or printing a 3D object.

[0019] The processor can receive or access the digital 3D model. The digital 3D model is typically displayed on a display screen in the form of a 3D mesh, representing the surfaces of the teeth and gingival tissue of the dental situation. The 3D mesh can consist of individual facets, e.g., triangular facets, while each facet can comprise, e.g., three interconnected vertices. Alternatively, the digital 3D model can be displayed as a point cloud comprising points, a graph comprising nodes and edges, a volumetric representation comprising voxels, or any other suitable 3D representation.

[0020] According to one embodiment, the method can comprise receiving a first digital 3D model representing a dental situation at a first point in time. A lesion of the dental situation, e.g., a caries lesion, can be identified on the first digital 3D model.

[0021] The method can further comprise receiving a second digital 3D model representing the dental situation at a second point in time.

[0022] The second time can be later than the first time, or can be earlier than the first time. In the exemplary case described herein, the second time is later than the first time. A lesion of the dental situation, e.g., a caries lesion previously identified, can also be identified on the second digital 3D model. However, the lesion of the dental situation identified on the second digital 3D model can be different from the lesion of the dental situation identified on the first digital 3D model. For example, the severity of the lesion can have changed over the period between the first time and the second time.

[0023] The method can additionally comprise generating a superimposed digital 3D model by aligning the first digital 3D model and the second digital 3D model to each other. Aligning the first digital 3D model and the second digital 3D model can comprise globally aligning the first digital 3D model and the second digital 3D model and / or locally aligning the first digital 3D model and the second digital 3D model. As a result of the local alignment, corresponding teeth of the first digital 3D model and the second digital 3D model are aligned to each other more accurately than in the global alignment. Aligning the first digital 3D model and the second digital 3D model can comprise, e.g., using an iterative closest point (ICP) algorithm and / or using individual tooth coordinate systems. Corresponding teeth of the first digital 3D model and the second digital 3D model can refer to teeth having the same universal numbering system (UNN) dental notation.

[0024] The method according to the present disclosure can comprise detecting a command issued by a user for interacting with one of the first digital 3D model or the second digital 3D model, applying the command to said one of the first digital 3D model or the second digital 3D model and automatically applying a corresponding command to the other of the first digital 3D model or the second digital 3D model. In this way, the first digital 3D model and the second digital 3D model can remain superimposed during user interaction. For example, the command can be a command for rotation, translation, view pan and / or zoom of the digital 3D model.

[0025] The method can further comprise displaying a portion of the superimposed digital 3D model comprising a lesion of a dental condition identified on the first digital 3D model and the second digital 3D model.

[0026] The display of the portion of the superimposed digital 3D model can occur based on user interaction with the superimposed digital 3D model, for example, when the user clicks on a lesion of the superimposed digital 3D model, the corresponding portion of the superimposed digital 3D model can be zoomed in. Upon zooming in, the superimposed digital 3D model can be automatically adjusted so that the lesion is in the view direction of the user. This view direction can be referred to as the ideal view direction.

[0027] The display of the portion of the superimposed digital 3D model comprising a lesion of a dental condition can comprise arranging a user-adjustable controller to vertically divide the displayed portion of the superimposed model into a first region and a second region, wherein only a portion of the first digital 3D model of the superimposed digital 3D model is rendered in the first region, and wherein only a portion of the second digital 3D model of the superimposed digital 3D model is rendered in the second region. In this way, the user can make an assessment for a specific lesion to determine changes in the lesion over time.

[0028] The user-adjustable controller can be movable by the user on the graphical user interface, for example, in left and right directions, to adjust the rendering of the first digital 3D model and / or the second digital 3D model of the superimposed digital 3D model. The user can switch between the rendering of a lesion of a dental condition identified on the first digital 3D model and the rendering of a lesion of a dental condition identified on the second digital 3D model by adjusting the user-adjustable controller. Due to its nature and effect of movement, the user-adjustable controller can also be referred to as a “slider” in the present disclosure.

[0029] The slider can move between two positions. The leftmost position of the slider renders only the lesion of the dental condition on the second digital 3D model. The rightmost position of the slider renders only the lesion of the dental condition on the first digital 3D model. Any position of the slider between the two positions renders a portion of the lesion in both the first and second digital 3D models.

[0030] The portion displaying the overlay of a digital 3D model of a lesion, including a dental condition, may further include identifying a first endpoint of the lesion in a first region and a second endpoint of the lesion in a second region. The first endpoint may be the leftmost point of the lesion in the first digital 3D model visible on the graphical user interface. The second endpoint may be the rightmost point of the lesion in the second digital 3D model visible on the graphical user interface.

[0031] The leftmost and rightmost points can be points on the frontal facets of the lesion, thus facing the virtual camera. When identifying the first and second endpoints, any lesion facets not facing the virtual camera can be ignored. Alternatively, the leftmost and rightmost points can be determined using only the non-occluded control points of the lesion. This ensures that the identified first and second endpoints are directly visible to the user on the graphical user interface, as they are not hidden by any part of the overlaid digital 3D model.

[0032] The portion displaying the superimposed digital 3D model of the lesion, including the dental condition, may further include a first interactive window positioned at any point on the display device whose x-coordinate is less than or equal to the x-coordinate of the first endpoint of the lesion. The first interactive window may include a severity value of the lesion identified on the first digital 3D model. Furthermore, the portion displaying the superimposed digital 3D model of the lesion may include displaying a second interactive window on the display device at any point whose x-coordinate is greater than or equal to the x-coordinate of the second endpoint of the lesion. The second interactive window may include a severity value of the lesion identified on the second digital 3D model.

[0033] The x-coordinates of the first and second endpoints can refer to the x-axis values ​​of the two-dimensional XY coordinate system of the display device, for example, they are located at the midpoint of the display device.

[0034] The x-coordinate can refer to the x-coordinate of the horizontal x-axis of the XY coordinate system (such as the XY coordinate system of a display device). The x-axis can be orthogonal to the y-axis of the XY coordinate system. These two axes can intersect, for example, at the midpoint of the display device.

[0035] Throughout the publication, the terms “lesion” and “lesion of dental condition” are used interchangeably.

[0036] In this way, annotations with metadata about the lesion can be displayed in the form of first and second interactive windows relative to the three-dimensional lesion location, so as not to hinder the viewing of the lesion.

[0037] The first interactive window and the lesion of the first digital 3D model can be arranged with each other such that the first end point is the only contact point between the lesion of the first digital 3D model and the first interactive window, or such that the first end point is the closest point between the lesion of the first digital 3D model and the first interactive window. Similarly, the second interactive window and the lesion of the second digital 3D model can be arranged with each other such that the second end point is the only contact point between the lesion of the second digital 3D model and the second interactive window, or such that the second end point is the closest point between the lesion of the second digital 3D model and the second interactive window.

[0038] The first and second interactive windows can include a severity value of the corresponding lesion. In addition, the interactive windows can further include further description information about the corresponding lesion, for example, such as lesion type, lesion identification number, identification number of the affected tooth, lesion size (expressed in the form of covered surface area), and / or lesion depth.

[0039] The first end point can be the leftmost point of the lesion in the first region. Similarly, the second end point can be the rightmost point of the lesion in the second region.

[0040] According to the present disclosure, the first interactive window can be displayed outside the lesion boundary of the first digital 3D model. Similarly, the second interactive window can be displayed outside the lesion boundary of the second digital 3D model. In this way, the interactive windows do not overlap with the displayed lesions and allow the user to view the lesions unobstructed.

[0041] The boundary of the lesion can be represented as a smooth spline including a plurality of control points. A process for identifying the first and second end points based on the plurality of control points of the lesion will be described below. How to transform the coordinates of the lesion from three-dimensional coordinates to two-dimensional coordinates using a transformation matrix will be described below.

[0042] Each of the plurality of control points of the lesion boundary can be transformed from a local three-dimensional space to normalized device coordinates (NDC) of a display device.

[0043] This transformation to the display device NDC can be performed in several steps. First, each of the plurality of control points of the lesion boundary is transformed from a local three-dimensional space to a world space coordinate system using a model matrix (M).

[0044] The local three-dimensional space can be a local coordinate system of the lesion. The world space coordinate system can be a global coordinate system of the digital three-dimensional scene. The model matrix can be a transformation matrix that translates, rotates, and / or scales the lesion so that the lesion is located in the world space coordinate system.

[0045] Next, each of the plurality of control points of the lesion boundary is transformed from the world space coordinate system to a view space coordinate system using a view matrix (V).

[0046] The view space coordinate system can be a coordinate system of a virtual camera representing the user’s view. The view matrix (V) can include translation and rotation operations to transform the plurality of control points of the lesion boundary to the front of the virtual camera.

[0047] Further, each of the plurality of control points of the lesion boundary can be transformed from the view space coordinate system to a clip space coordinate system using a projection matrix (P). In the clip space coordinate system, a coordinate range can be defined in order to clip and discard any coordinates outside of the defined range.

[0048] Further, each of the plurality of control points of the lesion boundary can be transformed from the clip space coordinate system to normalized device coordinates. For a viewport used to represent a position on a display device, the range of the normalized device coordinates can be from (0, 0, 0) to (1, 1, 1). The clip space coordinates can be converted to normalized device coordinates (NDC) by dividing each component by a homogeneous coordinate.

[0049] The sequence of transformations can accurately project the 3D points onto the 2D screen of the display device. The sequence of transformations from the local 3D space to the normalized device coordinates can be computed each time the digital 3D scene is updated.

[0050] Accordingly, the method according to the present disclosure can comprise updating the transformation of each of the plurality of control points of the lesion boundary from the local 3D space to the normalized device coordinates of the display device if the digital 3D scene including the displayed portion of the superimposed digital 3D model is updated.

[0051] Accordingly, the method can comprise detecting an update of the digital 3D scene and updating the transformation of each of the plurality of control points of the lesion boundary from the local 3D space to the normalized device coordinates of the display device.

[0052] In one embodiment, the previously described transformation procedure for each of the plurality of control points of the lesion boundary can be performed only for the control points of the lesion boundary that are not occluded. Accordingly, in one embodiment, each control point can be understood as each unoccluded control point. By only considering unoccluded control points, the identification of the first and second end points can be improved.

[0053] If a rotation, a translation, a view drag, a zoom in and / or a zoom out of the 3D scene is detected, the digital 3D scene including the displayed portion of the overlaid digital 3D model can be updated. These operations can be performed by the user during the interaction with the overlaid digital 3D model.

[0054] The method can further comprise identifying the first end point as the leftmost point of the lesion in the first region by identifying the lesion boundary point of minimum X coordinate in the normalized device coordinates.

[0055] The method can further comprise identifying the second end point as the rightmost point of the lesion in the second region by identifying the lesion boundary point of maximum X coordinate in the normalized device coordinates.

[0056] If the digital 3D scene including the displayed portion of the overlaid digital 3D model is updated, the rightmost point and / or the leftmost point can be repeatedly identified.

[0057] The first end point and the second end point, more specifically their x coordinate values, can define points at which the first interactive window and the second interactive window, respectively, can be placed. This placement will ensure that all the descriptive information about the lesion is always available to the user during the interaction with the overlaid digital 3D model, without covering any area of the lesion itself. Furthermore, the user can edit properties of the lesion, such as the severity of the lesion, in the first interactive window and / or the second interactive window while visually inspecting the lesion and / or comparing the lesion of the first digital 3D model and the lesion of the second digital 3D model.

[0058] As previously mentioned, the user can move the user-adjustable controller on the graphical user interface, for example in the left and right directions, to adjust the rendering of the first digital 3D model and / or the second digital 3D model of the overlaid digital 3D model. In this way, the rendering of the digital 3D model can be controlled by the user-adjustable controller. In addition to this, the movement of the user-adjustable controller can affect the rendering of the two-dimensional (2D) objects, such as the first interactive window and / or the second interactive window.

[0059] In one embodiment, if the user-adjustable controller is moved over the first interactive window, the method can comprise displaying only a portion of the first interactive window in the first region. In this way, if the slider is moved, for example, to cross a portion of the surface of the first interactive window, the user can observe that the portion of the first interactive window over which the slider is moved is hidden. The effect of this feature is that priority is given to rendering the second digital 3D model compared to rendering the entire first interactive window.

[0060] In one embodiment, if the user-adjustable controller is moved over the second interactive window, the method can comprise displaying only a portion of the second interactive window in the second region. In this way, if the slider is moved, for example, to span a portion of the surface of the second interactive window, the user can observe that the portion of the second interactive window over which the slider is moved is hidden. The effect of this feature is that priority is given to rendering the first digital 3D model compared to rendering the entire second interactive window.

[0061] The lesion can be identified by applying the trained neural network to the first digital 3D model and / or the second digital 3D model.

[0062] Typically, the trained neural network can be trained to detect the lesion of the dental condition directly on the digital 3D model of the tooth or on a two-dimensional image of the digital 3D model of the tooth.

[0063] In examples, the trained neural network can be a trained convolutional neural network (CNN) configured for detecting dental conditions such as dental caries, tooth wear, plaque, malocclusion, oral cancer, gingivitis, gum recession, etc. Separate trained neural networks can be used to detect different dental conditions. The trained neural network can be adapted not only to detect the presence of a dental condition on the digital 3D model but also to detect the severity of the dental condition on the digital 3D model.

[0064] The trained neural network can be trained on training data comprising annotated digital 3D models on which trained professionals can have marked the presence and / or severity of dental conditions.

[0065] The user can be able to modify the severity value of the lesion by interacting with the first interactive window and / or the second interactive window. Additionally or alternatively, the user can be able to modify other parameters of the lesion by interacting with the first interactive window and / or the second interactive window. For example, the user can enlarge or reduce the surface area of the lesion.

[0066] The method according to the present disclosure can comprise detecting the modified severity value of the lesion in the first interactive window and retraining the trained neural network based on the first digital 3D model and the modified severity value of the lesion. This advantageously allows the user to improve the behaviour of the trained neural network during its operation.

[0067] The method can additionally or alternatively comprise detecting the modified severity value of the lesion in the second interactive window and retraining the trained neural network based on the second digital 3D model and the modified severity value of the lesion.

[0068] Accordingly, the method can comprise transmitting the modified severity value and the corresponding digital 3D model to the input of the trained neural network in a feedback loop to retrain the trained neural network by the trained neural network.

[0069] In one embodiment, the retraining of the trained neural network can comprise:

[0070] - receiving the first digital 3D model and / or the second digital 3D model and a corresponding modified severity value of the lesion;

[0071] - generating an input for retraining the trained neural network based on the received first digital 3D model and / or the second digital 3D model and the corresponding modified severity value of the lesion;

[0072] - providing the input to the trained neural network and obtaining an output comprising a predicted severity value of the lesion;

[0073] - comparing the predicted severity value of the lesion with the modified severity value of the lesion to obtain a loss function value;

[0074] - updating the weights of the trained neural network based on the loss function value.

[0075] The input can comprise input elements (e.g. faces, points or vertices) of the first and / or second digital 3D model and geometry information related to the input elements.

[0076] The geometry information can comprise depth information related to a virtual camera position, an angle between a face normal and a virtual camera direction, face normal and / or curvature information.

[0077] The method according to the present disclosure can further comprise detecting a deletion of the lesion in the first interactive window and retraining the trained neural network based on the first digital 3D model and the deletion of the lesion. This can occur if the user assesses that the lesion was detected erroneously and decides to delete the lesion by interacting with the first interactive window.

[0078] Similarly, the method can comprise detecting a deletion of the lesion in the second interactive window and retraining the trained neural network based on the second digital 3D model and the deletion of the lesion.

[0079] The deletion of the lesion can also occur if a dental condition lesion is detected for a dental filling.

[0080] In one embodiment, the retraining of the trained neural network can comprise:

[0081] - receiving the first digital 3D model and / or the second digital 3D model and information of a deletion of a corresponding lesion;

[0082] - generating, based on the received first digital 3D model and / or second digital 3D model and the information of the deletion of the corresponding lesion, an input for retraining the trained neural network;

[0083] - providing the input to the trained neural network and obtaining an output containing a prediction of the presence of the lesion;

[0084] - comparing the prediction of the presence of the lesion with the information of the deletion of the corresponding lesion to obtain a loss function value;

[0085] - updating the weights of the trained neural network based on the loss function value.

[0086] According to the present disclosure, retraining the neural network advantageously allows the user to improve the behavior of the trained neural network while directly comparing the lesions of the first digital 3D model and the lesions of the second digital 3D model on the superimposed digital 3D model.

[0087] The method can further comprise adjusting a performance indicator of the trained neural network based on the detected modified severity value of the lesion. This performance indicator adjustment can be performed automatically if a modification of the severity value of the lesion is detected. The adjusted performance indicator can then be displayed.

[0088] The method can further comprise adjusting a performance indicator of the trained neural network based on the detected absence of the lesion. This performance indicator adjustment can be performed automatically if an absence of the lesion is detected. The adjusted performance indicator can then be displayed.

[0089] The method according to the present disclosure can comprise detecting a modified parameter of the lesion in the first interactive window and / or in the second interactive window and retraining the trained neural network based on the corresponding digital 3D model and the modified parameter of the lesion. The parameter of the lesion can be the surface area of the lesion, the volume of the lesion, the type of dental condition, etc.

[0090] Several techniques for tracking and visualizing changes in lesions will be discussed next.

[0091] The method can further comprise applying a first color and / or a first transparency value to the first digital 3D model and a second color and / or a second transparency value to the second digital 3D model, wherein the first color and / or the transparency value are different from the second color and / or the second transparency value.

[0092] In addition, a color gradient can be applied to the lesion to represent a change in the severity value of the lesion or the size of the lesion.

[0093] Instead of a color gradient, discrete color values can also be applied to the lesion to represent a change in the severity value of the lesion or the size of the lesion.

[0094] The method can further comprise generating a difference model from the first digital 3D model and the second digital 3D model.

[0095] The difference model can be obtained by subtracting the first digital 3D model from the second digital 3D model, or vice versa. Then, the first color can be applied to a growth area in the difference model, and the second color can be applied to a shrinkage area in the difference model.

[0096] The term“subtracting” can refer to determining a difference between corresponding vertices of the aligned first and second digital 3D models.

[0097] The corresponding vertices can be understood as the vertices in the aligned first and second digital 3D models that are closest to each other.

[0098] Additionally or alternatively, the difference model can be obtained by determining a difference between a surface area of the lesion in the first digital 3D model and a surface area of the lesion in the second digital 3D model. If the dental condition is gingival recession, the difference model can also relate to a distance between a point on the actual gingiva and a point on the ideal gingiva.

[0099] A plot of the lesion indicator over time can be generated, where the time is a time difference between the second time and the first time. The lesion indicator can be a surface area of the lesion, a depth of the lesion, and / or a severity value of the lesion.

[0100] Certain changes in the lesion can be considered normal and within a range expected to occur due to natural passage of time. For example, a certain amount of tooth wear can be normal over time, but there can be a threshold value of maximum amount of tooth wear expected over a period of time. Thus, the method can comprise displaying a reminder upon determining that the lesion indicator exceeds a predefined threshold value.

[0101] Furthermore, a transition animation between the lesion in the first digital 3D model and the lesion in the second digital 3D model can be generated and displayed.

[0102] Furthermore, the method can further comprise highlighting the changes between the lesion in the first digital 3D model and the lesion in the second digital 3D model during the display of the animation.

[0103] Highlighting the changes can comprise applying different color and / or texture values to the lesion in the first digital 3D model and the lesion in the second digital 3D model during the display of the animation.

[0104] The dental condition can be caries, plaque, tartar, oral cancer, gingival recession, tooth wear, gingivitis, and / or tooth fracture.

[0105] The present invention also discloses a computer readable medium comprising instructions which, when executed by a computer, cause the computer to perform a method according to the present disclosure.

[0106] According to the present disclosure, the present invention also discloses a computer program product comprising instructions which, when the program is executed by a computer, cause the computer to perform any method according to the present disclosure.

[0107] The present invention also discloses a dental scanning system comprising a data processing device configured to perform a method according to the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0108] Figure 1 A first digital 3D model of a dental situation and a second digital 3D model of the same dental situation are schematically shown;

[0109] Figure 2 A portion of the superimposed digital 3D models is shown in which a lesion of a dental situation is divided into two areas by a user-adjustable controller;

[0110] Figure 3 is a flowchart showing the steps of a method according to the present disclosure;

[0111] Figure 4 Steps of identifying a first and a second end point of a lesion when the digital 3D scene is updated to place an interactive window are shown;

[0112] Figure 5 A trained neural network for detecting a dental situation is shown, wherein there is a feedback loop for retraining the trained neural network;

[0113] Figure 6 is a flowchart showing the neural network training process;

[0114] Figure 7 A portion of the superimposed digital 3D models is shown in which a lesion of a dental situation is divided into two areas, the two areas having different severity values in each of the two areas;

[0115] Figure 8 An intraoral scanning system is shown. DETAILED DESCRIPTION

[0116] The following description will refer to the drawings, which show diagrammatically how the invention can be implemented.

[0117] Figure 1A first digital three-dimensional (3D) model 100 representing a dental condition of a patient at a first time, and a second digital three-dimensional (3D) model 101 representing the same dental condition at a second time, are shown. The second time can be later than the first time, or can be earlier than the first time. For illustrative purposes, the second time is chosen to be later than the first time. The first digital three-dimensional model 100 can correspond to the patient's teeth at a first visit to a dental clinic, while the second digital three-dimensional model 101 can correspond to the patient's teeth at a second visit to the dental clinic.

[0118] The first digital three-dimensional model 100 and the second digital three-dimensional model 101 can be different. The difference can reflect on the manifestation and / or progression of one or more dental conditions during the time period between the first time and the second time. Such dental conditions can be, for example, tooth wear, shown as dark areas on the first 3D model 100 and the second 3D model 101. These dark areas represent loss of tooth material due to tooth wear. Thus, there can be geometric differences between the first and second digital 3D models 100, 101. The geometric differences can be due to changes in tooth morphology caused by tooth wear. Figure 1 The upper jaw of the patient is shown, but the first and / or second digital 3D models 100, 101 can additionally or alternatively include the lower jaw, or include both the upper and lower jaws.

[0119] The first and second digital 3D models 100, 101 can have pairs of corresponding teeth. The corresponding teeth can be related to teeth having the same Universal Numbering System (UNN) symbol on the first and second digital 3D models 100, 101. For example, the canine tooth 102 on the first digital 3D model 100 has a corresponding canine tooth 104 on the second digital 3D model 101 with the same UNN symbol. Thus, Figure 1 The tooth 102 and its corresponding tooth 104 shown in FIG. 1 are examples of corresponding teeth.

[0120] It can be observed that the dental condition lesion (in this case, a tooth wear lesion) identified on the tooth 102 has not changed during the time period between the visits to the dental clinic, as the size and severity of the lesion is the same as on the corresponding tooth 104. However, the lesion 106 on the tooth 103 of the first digital 3D model can have changed during the time period between the visits to the dental clinic, as Figure 1 The global view of FIG. 1 can indicate an increase in the size and severity of the same lesion 106 on the corresponding tooth 105 of the second digital 3D model 101.

[0121] From the user's point of view, there is a need for an efficient comparison technique to compare in detail each of the lesions identified in the dental situation to assess whether and how the lesions have changed during time. This is needed in order to be able to establish a correct diagnosis and to make a corresponding treatment. As Figure 1 shown, comparing corresponding lesions on two separately displayed digital 3D models is not efficient, as the user needs to invest effort to visually identify the corresponding lesions and assess their similarity.

[0122] The first digital 3D model 100 and the second digital 3D model 101 can be superimposed on each other. It would be beneficial to superimpose and display the two digital 3D models, as a more efficient tracking of changes between the 3D models can be performed.

[0123] If the user selects a particular lesion, for example by clicking on the lesion 106, the part of the superimposed digital 3D models including the lesion 106 can be brought into the user's field of view, allowing the user to examine the selected lesion 106 in more detail.

[0124] Figure 2 A display of a part of the superimposed digital 3D models with lesions 106 of the dental situation is shown, which part is divided into two areas by a user-adjustable controller 200.

[0125] The controller 200 can vertically divide the displayed part of the superimposed digital 3D models containing the lesion 106 into a first area 201 and a second area 202. The controller 200 can also simultaneously vertically divide the lesion 106. The position of the controller 200 relative to the displayed part of the superimposed digital 3D models can define the position where a part of the first digital 3D model 100 of the superimposed digital 3D models will be rendered, and the position where a part of the second digital 3D model 101 of the superimposed digital 3D models will be rendered. In the first area 201, only the part of the first digital 3D model 100 of the superimposed digital 3D models can be rendered. In the second area 202, only the part of the second digital 3D model 101 of the superimposed digital 3D models can be rendered.

[0126] As Figure 2 shown, the first area 201 can be a part of the graphical user interface defined by the position of the controller 200 on its right side. Correspondingly, the second area 202 can be a part of the graphical user interface defined by the position of the controller 200 on its left side.

[0127] The user can drag the controller 200 (e.g. by using a computer mouse or touchpad) and thereby simultaneously observe at least a part of the lesion 106 of the first digital 3D model 100 and a part of the lesion 106 of the second digital 3D model 101. In this way, the user can be provided with an efficient comparison tool, wherein only by dragging the controller 200 to the left or right side of the graphical user interface, any changes in the lesion 106 of the dental situation can be observed.

[0128] Figure 2 Further, a first interactive window 203 and a second interactive window 204 are shown. These interactive windows can be understood as 2D annotations related to the lesion 106. In these interactive windows, at least a severity value of the lesion 106 can be indicated. Further, information such as a tooth identification number and / or a dental situation identification number can be displayed.

[0129] Figure 2 Further, a timeline 205 is shown, which can indicate time instances at which the patient had his dental situation scanned. For example, a point 206 can correspond to a first time at which the first digital 3D model 100 represents the patient’s dental situation. A point 207 can correspond to a second time at which the second digital 3D model 101 represents the patient’s dental situation.

[0130] Figure 3 A flowchart of a method 300 according to the present disclosure is shown.

[0131] Step 301 shows receiving a first digital 3D model 100 and a second digital 3D model 101.

[0132] In step 302, a superimposed digital 3D model can be generated. The superimposed digital 3D model can be generated by aligning the first digital 3D model 100 and the second digital 3D model 101 to each other. Aligning the first digital 3D model 100 and the second digital 3D model 101 can comprise globally aligning the first digital 3D model 100 and the second digital 3D model 101 and / or locally aligning the first digital 3D model 100 and the second digital 3D model 101. As a result of the local alignment, the corresponding teeth of the first digital 3D model 100 and the second digital 3D model 101 are more accurately aligned to each other than in the global alignment. Aligning the first digital 3D model 100 and the second digital 3D model 101 can comprise using an Iterative Closest Point (ICP) algorithm and / or using separate tooth coordinate systems. The corresponding teeth of the first digital 3D model 100 and the second digital 3D model 101 can refer to teeth having the same Universal Numbering System (UNN) dental symbol.

[0133] Step 303 shows displaying a portion of the superimposed digital 3D model, which includes the lesion 106 of the dental condition identified on the first digital 3D model 100 and the second digital 3D model 101.

[0134] The displaying step 303 can comprise arranging the user-adjustable controller 200 to vertically divide the displayed portion of the superimposed model into a first region 201 and a second region 202, wherein only a portion of the first digital 3D model 100 of the superimposed digital 3D model is rendered in the first region 201, and wherein only a portion of the second digital 3D model 101 of the superimposed digital 3D model is rendered in the second region.

[0135] The user-adjustable controller 200 can also vertically divide the lesion 106.

[0136] Step 305 represents identifying a first end point 208 of the lesion 106 in the first region 201. In step 306, a second end point 209 of the lesion 106 in the second region 202 can be identified.

[0137] Step 307 shows arranging a first interactive window 203 at the first end point 208 of the lesion 106, which contains the severity value of the lesion 106 identified on the first digital 3D model 101. The first interactive window 203 can be arranged at any point of the display device whose x-coordinate is less than or equal to the x-coordinate of the first end point 208.

[0138] Step 308 shows displaying a second interactive window 204 at the second end point 209 of the lesion 106, which contains the severity value of the lesion 106 identified on the second digital 3D model 101. The second interactive window 204 can be arranged at any point of the display device whose x-coordinate is greater than or equal to the x-coordinate of the second end point 209.

[0139] Figure 4 The steps of identifying the first end point 208 and the second end point 209 of the lesion 106 for placing the interactive windows 203 and 204 when updating the digital 3D scene are shown. The end points 208 and 209 can be dynamically calculated each time the user interacts with the superimposed digital 3D model, thus updating the digital 3D scene. As a result, this can enable the interactive windows 203 and 204 to be always located at the respective edges of the lesion 106.

[0140] Step 400 indicates that the process of identifying the first end point 208 and the second end point 209 can be repeated each time the digital 3D scene is updated.

[0141] In step 401, a model matrix (M) can be used to transform each of the plurality of control points of the lesion 106, more specifically the plurality of control points of the lesion 106 boundary, from a local 3D space to a world space coordinate system.

[0142] The local 3D space can be a local coordinate system of the lesion 106. The world space coordinate system can be a global coordinate system of the digital 3D scene. The model matrix (M) can be a transformation matrix that translates, rotates and / or scales the lesion 106 to place it in the world space coordinate system.

[0143] Step 402 illustrates the use of a view matrix (V) to transform each of the plurality of control points of the lesion 106 boundary from the world space coordinate system to a view space coordinate system. The view space coordinate system can be a coordinate system of a virtual camera representing the user’s view. The view matrix (V) can include translation and rotation operations to transform the plurality of control points of the lesion boundary to the front of the virtual camera.

[0144] In step 403, the use of a projection matrix (P) is illustrated to transform each of the plurality of control points of the lesion 106 boundary from the view space coordinate system to a clip space coordinate system. In the clip space coordinate system, a coordinate range can be defined such that any coordinates outside the defined range are clipped and discarded.

[0145] Step 404 illustrates the transformation of each of the plurality of control points of the lesion 106 boundary from the clip space coordinate system to normalized device coordinates. The range of the normalized device coordinates can be from (0, 0, 0) to (1, 1, 1) for a viewport representing a position on a virtual plotting device. The clip space coordinates can be converted to normalized device coordinates (NDC) by dividing each component by the homogeneous coordinate.

[0146] From this, steps 401-404 illustrate how each of the plurality of control points of the lesion 106 boundary can be transformed from a local 3D space to normalized device coordinates (NDC) of a display device. This sequence of transformations can accurately project the 3D points onto the 2D screen of the display device. The sequence of transformations from the local 3D space to the normalized device coordinates can be computed each time the digital 3D scene is updated.

[0147] Step 405 illustrates the identification of the first end point 208 and the second end point 209. The first end point 208 can be identified by locating the leftmost point of the lesion 106 in the first region 201. This can be achieved by identifying the point with the minimum x-coordinate of the lesion 106 boundary in the normalized device coordinates.

[0148] By identifying a point of the x-coordinate maximum of the lesion 106 border in the normalized device coordinates, the second end point 209 can be identified as the rightmost point of the lesion 106 in the second region 202.

[0149] The rightmost point and / or the leftmost point can be identified each time the digital 3D scene containing the displayed part of the superimposed digital 3D model is updated.

[0150] The first end point 208 and the second end point 209 can be used to position the first interactive window 203 and the second interactive window 204, respectively.

[0151] The user can interact with the first interactive window 203 and / or the second interactive window 204 and modify the severity value of the lesion 106 when visually inspecting the lesion 106.

[0152] The modified properties of the lesion can be used to retrain the trained neural network used to identify lesions 106 of dental conditions.

[0153] Figure 5 A trained neural network 500 for detecting dental conditions is shown, which has a feedback loop 508 for retraining the trained neural network 500.

[0154] The trained neural network 500 can be a convolutional neural network and can comprise a plurality of convolutional layers 502 capable of capturing low-level features of the input 501, i.e. obtaining convolutional features. In addition, there can also be one or more pooling layers 503 responsible for reducing the spatial size of the convolutional features. Convolutional neural networks are particularly suitable for processing matrix information, such as two-dimensional images.

[0155] The input 501 can be in two-dimensional or three-dimensional format and can comprise at least one input element, such as a tooth, a face sheet of a tooth, a point of a tooth point cloud and / or a voxel. Figure 5 An input is shown as a digital three-dimensional model of a jaw. The at least one input element can be a two-dimensional image of a tooth of the jaw, one of the surfaces of which is affected by a dental condition. The at least one input element can alternatively comprise one or more two-dimensional images of the tooth surface, or at least one pixel of the tooth two-dimensional image.

[0156] The output 505 can comprise at least one probability value representing the likelihood of the presence of a dental condition in the input 501 and / or can further classify the dental condition into a severity class. For example, a lesion of caries can be detected in the input 501 and assigned a moderate severity value. The severity value can also be mild or severe, for example.

[0157] Figure 5Block 506 is further shown, which illustrates the detection of the user's modification of the output 505. The user can compare the lesion 106 of the first digital 3D model 100 with the lesion 106 of the second digital 3D model 101, for example in the tools shown. The user can interact with the first interactive window 203 and / or the second interactive window 204 and modify the severity value of the lesion 106. For example, the user can change the severity value of the lesion 106 from "moderate" to "severe" in the second interactive window. Figure 2

[0158] Once the modification of the output 505 is detected, a feedback loop 508 can be introduced to retrain the trained neural network 500 based on the modification of the output 505. The trained neural network 500 can transmit the modified severity value and the corresponding digital 3D model to the input 501 of the trained neural network 500 in the feedback loop 508 to retrain the trained neural network 500.

[0159] During the retraining of the trained neural network 500, the modified severity value can be used as a ground truth and the weights of the trained neural network 500 can be adjusted by comparing the output of the trained neural network with this ground truth.

[0160] The output modification 506 can include the detection of the deletion of the lesion 106 in the first interactive window 203 and / or the second interactive window 204. For example, if the trained neural network 500 indicates that there is a lesion 106 on a dental filling, the user can decide to delete the lesion 106. This information can also be used to retrain the trained neural network 500 through the feedback loop 508.

[0161] Figure 6 A flowchart of the training process of the neural network for detecting lesions 106 of dental conditions is shown.

[0162] The neural network can be trained for a classification task. The training process of the classification task of the trained neural network 500 is described as follows. The overall goal of the untrained neural network is to develop a trained neural network 500 that can classify the input 501 according to the presence of lesions of dental conditions and / or the severity value of the lesions.

[0163] ​The training process can be referred to as "supervised training" due to the use of labeled training data. Step 601 illustrates obtaining training data. The training data can contain at least 1000 three-dimensional digital models, which can generate over 17000 individual teeth. The training data can be in 2D format, i.e. it can comprise a plurality of 2D training images of teeth or tooth parts. The training data can also be in 3D format, comprising a plurality of teeth in the form of 3D meshes, 3D graphs or point clouds.

[0164] Step 602 illustrates generating target data from the obtained training data. This can refer to annotating the training data to obtain ground truth data. Thus, the target data can comprise labels indicating the presence of dental conditions on the training data. The annotation, also referred to as labeling, can be performed by, for example, qualified dental practitioners. The training data can be labeled according to the presence of dental conditions, for example using "yes" or "no" labels to indicate the presence or absence of a dental condition. Additionally or alternatively, the training data can be labeled using a severity value label related to the corresponding lesion.

[0165] These labels can be considered as targets or ground truths for a given training data set.

[0166] The training data can be fed into an untrained neural network, as illustrated by step 603. The output of the untrained neural network can be obtained, which contains probability values indicating the output class to which the input or input elements belong.

[0167] The loss can be determined by comparing the output with the generated target data, as illustrated by step 604. The loss can be determined by a loss function. One example of a loss function can be a mean squared error function. In order to train the untrained neural network, this loss function should be minimized. The minimization of the loss function can be achieved, for example, by using a stochastic gradient descent algorithm.

[0168] By this process of minimizing the loss function, the weights of the untrained neural network can be updated, as illustrated by step 605.

[0169] The training process can be an iterative process, in which the same training data set can be iteratively fed into the untrained neural network until a stopping criterion is reached, as illustrated by step 606. The stopping criterion can be a certain threshold value of the loss function. The stopping criterion can be a number of training epochs, after which the performance metric no longer increases.

[0170] Figure 7 A portion of the superimposed digital 3D model is shown, which has a lesion 106 of a dental condition, which is divided into two regions 203, 204, which have different severity values in each of the two regions 203, 204.Figure 7 The same elements are shown, with the difference that the lesions 106 are generated in a different way. Figure 2

[0171] First, in Figure 7 the second region 202, and thus in the second digital 3D model 101 of the superimposed digital 3D model, the lesions 106 of the dental condition are marked as having a severity degree of "severe", while the lesions 106 in the first region 201 have a severity degree of "moderate". These severity degrees can be assigned by the trained neural network 500, or can be manually adjusted by the user by interacting with the first and / or second interactive windows 203, 204.

[0172] Furthermore, Figure 7 It is also shown how the lesions 106 and the internal surface defined by the lesion boundaries can be presented to provide an efficient comparison tool.

[0173] A first color and / or a first transparency value can be applied to the first digital 3D model 100 or to the lesions 106 of the first digital 3D model 100, while a second color and / or a second transparency value can be applied to the second digital 3D model 101 or to the lesions 106 of the second digital 3D model 101. The first color and / or transparency value can be different from the second color and / or second transparency value.

[0174] Alternatively or additionally, a color gradient or discrete color values can be applied to the lesions 106 to represent a variation in the severity value of the lesions 106 or in the size of the lesions 106.

[0175] It can be introduced Figure 7 ​Other features not shown can be displayed to improve the efficiency of the comparison tool. For example, a difference model can be generated from the first digital 3D model 100 and the second digital 3D model 101 by subtracting the first digital 3D model 100 from the second digital 3D model 101, or vice versa. Then, a first color can be applied to the growth areas in the difference model, and / or a second color can be applied to the shrinkage areas in the difference model. In addition, an indicator of the lesion 106 over time can be plotted, where the time is the time difference between the second time and the first time. The indicator of the lesion 106 can be the surface area or severity value of the lesion 106. Additionally or alternatively, a transition animation between the lesion 106 in the first digital 3D model 100 and the lesion 106 in the second digital 3D model 101 can be generated and displayed. Thereby, the changes between the lesion 106 in the first digital 3D model 100 and the lesion 106 in the second digital 3D model 101 can be highlighted. Highlighting the changes can alternatively or additionally include applying different color and / or texture values to the lesion 106 in the first digital 3D model 100 and the lesion 106 in the second digital 3D model 101 during the display of the animation.

[0176] Figure 8 A dental scanning system 800 is shown, which can include a computer 810 capable of performing any of the methods of the present disclosure. The computer can include a wired or wireless interface to a server 815, a cloud server 820, and an intraoral scanner 801. The intraoral scanner 801 is capable of recording scanning data, including geometric information, natural color information, and / or fluorescence information related to a patient’s dentition. The intraoral scanner 801 can be equipped with various modules, such as a fluorescence module or an infrared module, thereby being capable of capturing information related to diagnosing dental conditions such as caries, gum recession, tooth cracks, gingivitis, and / or plaque.

[0177] The dental scanning system 800 can include a data processing device configured to perform a method according to one or more embodiments of the present disclosure. The data processing device can be part of the computer 810, the server 815, or the cloud server 820.

[0178] A non-transitory computer-readable storage medium or a computer-readable storage medium can be included in the dental scanning system 800. The non-transitory computer-readable medium can carry instructions that, when executed by a computer, cause the computer to perform a method according to one or more embodiments of the present disclosure.

[0179] A computer-readable storage medium can include optical storage media (e.g., optical discs), machine- readable barcodes, USB flash drives, magnetic storage devices, solid-state electronic memory devices (e.g., random-access memory (RAM) or read-only memory (ROM)), or any other physical device or medium employed to store computer programs. Thus, embodiments of the present disclosure can be used in data processing hardware apparatuses (e.g., computer systems or personal computers) or can be used in embedded systems employing special-purpose data processing units (e.g., digital signal processing chips).

[0180] A computer program product can be embodied in a non-transitory computer-readable storage medium. The computer program product can include instructions that, when executed by a computer, cause the computer to perform a method according to any of the embodiments presented herein.

[0181] References in the present disclosure to certain features, advantages or similar language do not imply that all of the features and advantages that can be meant to be included in any single embodiment. Rather, each feature or benefit that is described in connection with an embodiment should be considered as included in at least one embodiment within the scope of the present disclosure. Thus, the references to features, advantages and similar language (e.g., "advantage" or "feature") within this disclosure should be understood as meaning that a particular feature, advantage or property described can usefully be employed in at least one embodiment.

Claims

1. A computer-implemented method for evaluating a digital 3D model of a tooth, characterized in that, The method comprises: - receiving a first digital 3D model representing a dental situation at a first time; - receiving a second digital 3D model representing a dental situation at a second time; - generating a superimposed digital 3D model by aligning said first digital 3D model and said second digital 3D model to each other; - displaying a portion of said superimposed digital 3D model, said portion comprising a lesion of a dental condition identified on said first digital 3D model and said second digital 3D model, wherein said displaying comprises: - arranging a user-adjustable controller to vertically divide the displayed portion of the superimposed model into a first region and a second region, wherein only a portion of said first digital 3D model of said superimposed digital 3D model is rendered in said first region, and wherein only a portion of said second digital 3D model of said superimposed digital 3D model is rendered in said second region; - identifying a first endpoint of said lesion in said first region and a second endpoint of said lesion in said second region; - arranging a first interactive window at any point of the display device whose x-coordinate is less than or equal to the x-coordinate of the first endpoint of said lesion, said first interactive window comprising a severity value of said lesion identified on said first digital 3D model, wherein said first interactive window is displayed outside the boundaries of said lesion of said first digital 3D model; and - arranging a second interactive window at any point of the display device whose x-coordinate is greater than or equal to the x-coordinate of the second endpoint of said lesion, said second interactive window comprising a severity value of said lesion identified on said second digital 3D model, wherein said second interactive window is displayed outside the boundaries of said lesion of said second digital 3D model.

2. The method of claim 1, wherein, The first endpoint is the leftmost point of said lesion in said first region, and the second endpoint is the rightmost point of said lesion in said second region.

3. The method according to any of the preceding claims, characterized in that, The boundaries of said lesion are represented as a smooth spline comprising a plurality of control points.

4. The method of claim 3, wherein, The determination of the leftmost point of said lesion and the rightmost point of said lesion only considers non-occluded control points of said smooth spline and / or front-facing facets of said lesion.

5. The method according to claim 3 or 4, characterized in that, Further comprising transforming each control point of said plurality of control points of the boundaries of said lesion from a local 3D space to normalized device coordinates (NDC) of a display device.

6. The method according to the preceding claim 5, characterized in that, Further comprising: if the digital 3D scene comprising the displayed portion of said superimposed digital 3D model is updated, updating the transformation of each control point of said plurality of control points of the boundaries of said lesion from said local 3D space to normalized device coordinates of said display device.

7. The method according to claim 5 or 6, characterized in that, Further comprising: identifying said first endpoint as the leftmost point of said lesion in said first region by identifying the point in normalized device coordinates whose x-coordinate value is the smallest of the boundaries of said lesion.

8. The method according to any of the preceding claims 5-7, characterized in that, Further comprising: identifying said second endpoint as the rightmost point of said lesion in said second region by identifying the point in normalized device coordinates whose x-coordinate value is the largest of the boundaries of said lesion.

9. The method according to any of the preceding claims, characterized in that, If the user-adjustable controller is moved over the first interactive window, only a portion of the first interactive window is displayed in the first area.

10. The method according to any of the preceding claims, characterized in that, If the user-adjustable controller is moved over the second interactive window, only a portion of the second interactive window is displayed in the second area.

11. The method according to any of the preceding claims, characterized in that, identifying the lesion by applying a trained neural network to the first digital 3D model.

12. The method of claim 11, wherein, Further comprising: detecting a modified severity value of the lesion in the first interactive window and retraining the trained neural network based on the first digital 3D model and the modified severity value of the lesion.

13. The method according to any of the preceding claims, characterized in that, Further comprising generating a difference model from the first digital 3D model and the second digital 3D model by subtracting the first digital 3D model from the second digital 3D model or subtracting the second digital 3D model from the first digital 3D model, wherein subtracting comprises determining a difference between corresponding vertices of the aligned first digital 3D model and the second digital 3D model.

14. A computer readable medium containing instructions, characterized in that, The instructions, when executed by a computer, cause the computer to perform the method according to any of the preceding claims 1-13.

15. A computer program product comprising instructions, characterized in that, The instructions, when executed by a computer, cause the computer to perform the method according to any of the preceding claims 1-13.

Images