Method for superimposing digitized images and reference marker device

Abstract

Method for superimposing digitized representations of a human body region, wherein a digitized three-dimensional first data set of a first body region model and a digitized three-dimensional second data set of a second body region model are displayed on a display device (1), wherein a matching reference system (5) for displaying the first data set and the second data set is generated on the basis of first control points within the first data set and second control points within the second data set, wherein a reference marker device (4, 14) is formed onto the human body in a transition region between the first body region model and the second body region model, wherein a first auxiliary data set of the first body region model is displayed together with the first auxiliary reference markers by the reference marker device (4,14) and a second auxiliary data set of the second body region model is generated together with the second auxiliary reference markers by the reference marker device (4, 14), and wherein a matching reference system (5) for the first auxiliary data set and the second auxiliary data set is generated by comparing the spatial arrangement of the first auxiliary reference markers and the second auxiliary reference markers, characterized in that the first data set is adapted to the first auxiliary data set and the second data set is adapted to the second auxiliary data set.

Description

[0001] The invention relates to a method for superimposing digitized representations of a human body region, wherein a digitized three-dimensional first data set of a first body region model and a digitized three-dimensional second data set of a second body region model are displayed on a display device, wherein a matching reference system for displaying the first data set and the second data set is generated on the basis of first control points within the first data set and second control points within the second data set, wherein a reference marker device is formed on the human body in a transition region between the first body region model and the second body region model,wherein a first auxiliary data set of the first body region model is generated together with the first auxiliary reference markers by the reference marker device, and a second auxiliary data set of the second body region model is generated together with the second auxiliary reference markers by the reference marker device, and wherein a matching reference system for the first auxiliary data set and the second auxiliary data set is generated by comparing the spatial arrangement of the first auxiliary reference markers and the second auxiliary reference markers.

[0002] In cosmetic or medical procedures involving the treatment, modification, or replacement of body parts, digital modeling of the treatment procedure or the modified or replaced body part can be helpful. Using a model, both the physician and the patient can simulate, visually perceive, and verify the progress and outcome of the treatment, or individually adjust it to achieve the desired treatment outcome.

[0003] For example, if a prosthesis or dental prosthesis is to be manufactured, fitted to the patient and connected to the patient, a body area model of the affected body area and a model of the prosthesis or dental prosthesis or dentures can be created and displayed superimposed on a display device in order to simulate and visually display the appearance and functional possibilities of the prosthesis or dental prosthesis.

[0004] Particularly in the manufacture and attachment of fixed denture components and implants, the desired treatment success can depend significantly on the most precise and individual adaptation of the shape of the denture components or implants to the patient's remaining teeth or jaw and on the shape and appearance of the denture component or implant being adapted as well as possible to adjacent areas of the mouth and to the patient's appearance.

[0005] For this purpose, it is known in practice to generate a digitized three-dimensional first data set of a first body region model, which can be visually displayed. In addition, a digitized three-dimensional second data set of a second body region model is generated and also displayed on the display device. The first body region model can represent a damaged limb, while the second body region model represents a prosthesis to be connected to the damaged limb. In the field of dentistry, the first body region model can represent the head, and the second body region model can represent a set of teeth with or without reconstructive measures or dentures.

[0006] The first body region model is typically created using an optical scanning device suitable for generating a first data set for a body region model of the affected body region with sufficient resolution and accuracy. The second body region model can be created using the same scanning device. If the second body region model represents a prosthesis, it is conceivable to generate the second body region model exclusively with computer support. If the second body region model has significantly smaller dimensions than the first body region model and represents, for example, a denture or a set of teeth, it is often expedient to capture the second body region model using an optical scanning device that is better suited for objects with small dimensions.Particularly suitable dental scanners have been developed for dental models and dentures, which enable high resolution and positional accuracy of the digitized dental model.

[0007] In order to display the first body region model and the second body region model correctly arranged and aligned relative to each other on the display device, it is necessary to embed the corresponding data sets or body region models in a matching reference system with precise positioning and matching alignment. Subsequently, a simultaneous or superimposed representation of the two body region models can be generated using the matching reference system.

[0008] The matching reference system can be determined by comparing first control points within the first data set and second control points within the second data set, provided that the first control points and the second control points in the different data sets each represent identical positions.

[0009] When embedding a tooth model in a head model, it is known from practice that an experienced operator manually creates a number of first control points in the first body region model by selecting individual dataset elements or marking model points. Similarly, a second number of second control points is also manually created in the second body region model, with the first control points and the second control points spatially matching as precisely as possible. Depending on the operator's skill, this can usually be achieved with an accuracy of approximately 1 mm to 2 mm.

[0010] The two body region models are then transferred to a common reference system, and their respective positions and orientations are modified and specified to achieve the best possible match when displaying the two body region models. Particularly in dentistry and dental technology, this arrangement and alignment of the individual models relative to one another is referred to as registration. The mathematical methods required for this, which can include spatial operations such as translation or rotation, as well as linear or nonlinear optimization algorithms such as least-square fits or other numerical optimization methods, are known in many variations and can, if necessary, be used as prefabricated modules and integrated into customized software solutions.

[0011] It has been shown that the manual selection and digital definition of first and second control points in the respective representations of the body region models is time-consuming and labor-intensive. Despite considerable operator experience and careful operation, it is rarely possible to generate an error in the superposition of the two body region models that is less than 1 mm offset or less than a few degrees of misalignment to each other.

[0012] In principle, it is possible to identify further striking shapes or patterns in the two body region models in addition to the manually specified control points with the help of suitable software and use them as additional control points. It has been shown that clearly identifying additional control points without operator interaction is often problematic, especially since it cannot be guaranteed that all areas potentially suitable for additional control points are visible in both body models. A comparison of the additional control points generated automatically often does not lead to a noticeable improvement in accuracy and, due to the many sources of error, should be verified by a subsequent manual review of the automatically obtained results, which creates additional effort.

[0013] It is common practice to use a consistently positioned reference point aid with multiple control points when creating the two body region models separately. The control points are automatically captured during digitization and the creation of the two data sets and can then be used to convert the two data sets into a common reference system. In dentistry, a denture fork is used for this purpose. This is positioned in the mouth during the acquisition of a head model and positioned on the dentures for the acquisition of the denture model.A section of the denture protruding from the mouth provides suitable control points, but also obscures corresponding head and dentition areas. Therefore, the optically recorded and subsequently digitized head and dentition models are incomplete due to the obscured areas, making the desired visualization of a planned dental procedure, such as tooth correction or an implant, difficult or even impossible. Such a method is disclosed, for example, in JP 2009 233 294 A.

[0014] It is therefore considered an object of the present invention to facilitate the embedding of two different body region models into a common consistent reference system, to allow high accuracy and to enable an appealing and informative visualization.

[0015] This object is achieved according to the invention by adapting the first data set to the first auxiliary data set and the second data set to the second auxiliary data set. By using a reference marker device, the first and second auxiliary reference markers required for registration or for determining a matching reference system can be generated without manual intervention.

[0016] The auxiliary reference markers are generated by additional scanning processes of the first body region and the second body region in a first auxiliary data set and in a second auxiliary data set. In these auxiliary data sets, the areas obscured by the reference marker device are not digitized; this, as well as the additional scanning processes required, is accepted. By molding the reference marker device to the body region, it can be repeatedly applied to the transition region with great precision in order to generate the individual auxiliary data sets. Since the auxiliary reference markers are located very precisely in the same spatial location in both auxiliary data sets, a consistent reference system for both auxiliary data sets can be determined with great precision.Subsequently, only the first body region model and the second body region model, which were generated without the use of the reference marker device, need to be adapted to the respectively assigned first and second auxiliary data sets in order to be displayed on the display device in the correct position and alignment with respect to one another. This can be done using a method known from practice for adapting body region models or model data and using automatically generated control points within the body region models. These control points were not generated using the reference marker device, but are manually specified control points or, preferably, conspicuous shapes or patterns in the two body region models identified with the aid of suitable software.

[0017] Preferably, the first data set and the first auxiliary data set of the first body region model are generated with a first optical digitization device, and the second data set and the second auxiliary data set of the second body region model are generated with a second optical digitization device. With the aid of the reference marker device, data sets generated in different digitization devices can be easily and automatically converted into a consistent reference system.

[0018] For example, if the first body region model is a head model and the second body region model is a tooth model, the head model can be created using a body scanner or an object scanner suitable for sufficiently large objects, while the tooth model is created using a tooth scanner or a denture scanner. While the body scanner often does not specify an orientation of the body relative to the scanning device, the dentures can be positioned and aligned in a predetermined manner within a housing of the tooth scanner. Furthermore, the resolutions and accuracies of the various digitization devices differ from one another in order to be able to generate the smallest possible data sets as quickly as possible, adapted to the object, while allowing sufficient detail and graphic resolution for visualization.

[0019] According to an advantageous embodiment of the inventive concept, it is provided that, based on a plurality of spaced-apart and non-rotationally symmetrically arranged first and second auxiliary reference markers, a reference system is determined for the first auxiliary data set and for the second auxiliary data set that corresponds as closely as possible to the first and second auxiliary reference markers. A large distance between the individual auxiliary reference markers enables great precision in the comparison of the auxiliary reference markers in the various auxiliary data sets. An asymmetrical arrangement allows the orientation of the body region models relative to one another to be clearly determined with little effort, without the need for additional input from an operator.

[0020] It is also conceivable to specify a rotationally symmetrical arrangement of the auxiliary reference markers, since in many cases the basic orientation of the body area models is already known or can be easily specified and the auxiliary reference markers are not intended to reveal the orientation, but merely to improve the accuracy of the relative positioning and alignment of the body area models to one another.

[0021] In order to prevent incorrect positioning of the reference marker device during potentially multiple scanning processes, the reference marker device is provided with an impression layer containing a pasty, hardening impression material and is molded onto the transition region of the body region to be imaged using the first and second body region models before hardening, and the first auxiliary data set and the second auxiliary data set are generated after the molded impression material has hardened. The reference marker device, which is adapted to the shape of the body in the transition region, no longer changes the shape of the hardened impression material in the relevant region of the impression layer. The reference marker device can therefore be repeatedly molded onto the body or a prosthesis, orapplied to a working model for a dental prosthesis and positioned precisely so that multiple auxiliary data sets can be generated and used to adapt the first body area model and the second body area model.

[0022] According to a particularly simple and quickly implementable method, it is provided that the reference marker device has an impression layer with an impression material that can be plastically deformed by a deformation pressure, which is deformed by pressure and shaped to the transition region of the body region to be imaged with the first and second body region models, and that the first auxiliary data set and the second auxiliary data set are generated without pressure being applied above the deformation pressure. At the reference marker device, a dental impression can be generated in the impression material of the impression layer, for example by placing the reference marker device in the mouth and biting down. The reference marker device can then be placed on a working model for a denture that has already been adapted to the patient in question or has been generated by a previous impression process with a bite fork.The dental impression in the impression layer ensures that the reference marker device is consistently positioned and aligned during repeated scanning processes both on the working model and in the patient's oral cavity.

[0023] Wax is a suitable impression material for many applications, especially for dental models. Other impression materials are also possible, for example, those that can be heated for shaping or chemically treated for hardening.

[0024] The invention also relates to the use of a reference marker device with a carrier element and with an impression layer arranged on the carrier element with a deformable impression material for carrying out the method described above.

[0025] A carrier element with an area in which an impression material is arranged is known from practice, for example, as a bite fork for the production of a dental impression or a working model for the production and adaptation of a dental prosthesis.

[0026] It is provided that at least one photogrammetric or surface-profiled reference marking is arranged on the carrier element. The reference marking serves to generate auxiliary reference markers during an optical scanning process in which the reference marker device is captured together with a body region or an object adjacent to the body region or to be embedded therein. These auxiliary reference markers enable easy determination of the position and orientation of the scanned data set in a reference system.

[0027] Preferably, the reference marker device comprises a reference plate on which the at least one reference marking is arranged. The reference plate expediently comprises a completely or partially flat surface on which the at least one reference marking is arranged. The reference plate forms a well-defined base or background for the reference markings, so that after an optical scanning process, the position and orientation of the reference marker device can be determined quickly and reliably. The reference plate can also have a curvature or a continuous and well-defined surface profile in order to approximately adapt the reference plate to the body surface and to keep the distance of the individual reference markings from the body surface small.The reference plate may have a uniformly colored surface on which the individual reference markings are clearly visible and can be precisely identified.

[0028] The reference plate covers an area behind it and forms a flat and uniform background regardless of the object being scanned.

[0029] A clear determination of position and orientation usually requires that the reference marker device has at least three symmetrical reference markings that are arranged asymmetrically. Additional reference markings can generate a mathematically overdetermined system of equations and enable a numerical adjustment of the measured values ​​to the known and predetermined geometry of the reference markings in order to improve the precision of the scanned or optically acquired data sets and their embedding in a common reference system. If more reference markings are used than are necessary to determine the common reference system, it can be accepted that a reference marking is inadvertently or inevitably obscured during individual scanning processes and cannot be taken into account for the evaluation.

[0030] The reference marking can have a complex shape, so that a single reference marking is sufficient for determining position and orientation. Particularly simple and reliable determination of the common reference system is facilitated by the reference marking being a color-contrasting dot or ring. The dot or ring expediently has a color that differs significantly from the surface of the reference plate or at least from the adjacent areas of the reference plate. Other photogrammetric reference markings may be equally suitable and, in individual cases, advantageous for a corresponding application.

[0031] Preferably, the reference marking has a high contrast range. A black ring on a white background is suitable for many applications. Other light shapes on a dark background, or dark shapes on a light background, can also be used as a reference marking. Furthermore, it may be expedient to use several different shapes for adjacent reference markings, for example, to further verify alignment and perform additional plausibility checks.

[0032] It is also possible to use a spatially protruding or recessed shape as a reference marker instead of a flat color contrast. The reference marker can be a surface with almost any profile, the shape of which should be known as precisely as possible for subsequent evaluation. With increasing profiling, the identification of individual auxiliary reference markers is facilitated and the determination of a common reference system is improved. It is also possible for the reference marker to have several protruding reference shapes arranged at a distance from one another. A suitable reference shape could, for example, be a sphere or cone, a tetrahedron, or a pyramid arranged at a distance from the reference table.

[0033] The following examples are explained in more detail, as shown in the drawing. It shows: Fig. 1 is a schematic view of a display device on which a head model and a denture model are superimposed in a common reference system, Fig. 2 a schematic representation of a process flow for creating a common reference system using a reference marker device and auxiliary data sets, Fig. 3 a perspective view of a reference marker device, Fig. 4 a side view of a differently designed reference marker device and Fig. 5 a view of the front of the Fig. 4 shown reference marker device.

[0034] Particularly in the field of cosmetic or medical treatments, it is advantageous if digital data sets for three-dimensional object representations generated with different recording devices can be displayed on a display device 1 in a consistent reference system. Fig. 1, a head model 2 with a denture model 3 embedded therein is displayed on the display device 1, which is merely schematically illustrated. If the denture model 3 is displayed in the exact position and with the correct alignment within the head model 2, the effects of reconstructive measures on individual teeth or of the use of an individually manufactured and fitted denture, for example, can be visualized.

[0035] Head model 2 is generated from a first data set and represents a first body region model. Dentition model 3 is generated from a second data set and represents a second body region model. Head model 2 was generated using a body scanner (not shown), while dentition model 3 was generated using a dental scanner (also not shown). Since the arrangement and orientation of the patient's head were not precisely specified during the acquisition of head model 2, the relative positioning and orientation of head model 2 to dentition model 3 is initially unknown.

[0036] Only by displaying the head model 2 and the denture model 3 in a matching reference system, which is used in the representation by the display device 1, can a meaningful and accurate visualization of the denture model 3 within the head model 2 be created, which can serve as a basis for reconstructive measures by the doctor or for the individual adaptation of a denture to the individual patient.

[0037] In Fig. 2 schematically shows a process sequence in which an automatic assignment of the head model 2 and the denture model 3 to a common, matching reference system 5 is determined with the aid of a reference marker device 4.

[0038] In a preparatory method step, the reference marker device 4 is molded to the dentition of a person (not shown) by taking a dental impression with the reference marker device 4. With the reference marker device 4 in the mouth, the person's head is optically scanned with a body scanner, and a first auxiliary data set containing first auxiliary reference markers is acquired by the reference marker device 4. The reference marker device 4 is then applied to a previously produced working model 6 of the person's dentition, which was also used to create the dental model 3. The correct position and alignment of the reference marker device 4 on the working model 6 is ensured by the dental impression molded with the reference marker device 4 and contained therein.The working model 6 and the adjacent reference marker device 4 are also scanned with a dental scanner, and a second auxiliary data set containing second auxiliary reference markers is generated. In a visualization, the first auxiliary data set would represent an auxiliary head model 7 and the second auxiliary data set an auxiliary denture model 8, with the reference marker device 4 in each case obscuring an area of ​​the underlying head model 2 or denture model 3.

[0039] In a subsequent method step A, the first auxiliary data set and the second auxiliary data set are transferred into the common and consistent reference system 5 by comparing the respective first and second auxiliary reference markers.

[0040] Subsequently, in a method step B, the head model 2 is compared with the auxiliary head model 7 and the denture model 3 with the auxiliary denture model 8 and also transferred into the common reference system 5. In this way, the head model 2 and the denture model 3 can be transferred into the common reference system 5 and displayed correctly positioned and aligned relative to one another on the display device 1, without the need for manual processing and adjustment of the individual data sets.

[0041] In Fig. 3 shows a perspective view of the reference marker device 4. The reference marker device 4 has a support device 9 in the form of a registration fork, which is comparable in shape and functionality to a bite fork, which is frequently used in reconstructive dentistry for the production of working models for the individual adaptation of dentures and implants.

[0042] A rectangular and flat reference plate 11 is attached to a front side 10 of the support device 9. Five ring-shaped reference markings 12 are arranged asymmetrically on the reference plate 11.

[0043] An impression layer 13 made of an initially pasty impression material that hardens within a short time is arranged in a recess at the top of the carrier device 9. The impression layer 13 serves to take a dental impression of the upper jaw of the person to be treated in order to be able to later arrange the reference marker device 4 either in the same location in the mouth or in the person's dentition after the reference marker device 4 has been removed from the person's mouth or dentition, or to be able to place the reference marker device 4 on the working model 6 in a position and orientation that exactly corresponds to the first position and orientation of the reference marker device 4 in the person's mouth.

[0044] In the Fig. 4 and Fig.5 shows a differently designed reference marker device 14 merely by way of example. The reference marker device 14 also has a fork-shaped or horseshoe-shaped support device 15, which is coated on one upper side with an impression layer 16 made of wax or a moldable impression material. Eight hemispherical, protruding reference protrusions 17 are arranged on the reference plate 11 arranged on the front side 10, each forming a reference mark. A flat handle 18 is formed on a lower edge of the reference plate 11.

Claims

[1] Method for superimposing digitized representations of a human body region, wherein a digitized three-dimensional first data set of a first body region model and a digitized three-dimensional second data set of a second body region model are displayed on a display device (1), wherein a matching reference system (5) for displaying the first data set and the second data set is generated on the basis of first control points within the first data set and second control points within the second data set, wherein a reference marker device (4, 14) is formed onto the human body in a transition region between the first body region model and the second body region model, wherein a first auxiliary data set of the first body region model is displayed together with the first auxiliary reference markers by the reference marker device (4,14) and a second auxiliary data set of the second body region model is generated together with the second auxiliary reference markers by the reference marker device (4, 14), and wherein a matching reference system (5) for the first auxiliary data set and the second auxiliary data set is generated by comparing the spatial arrangement of the first auxiliary reference markers and the second auxiliary reference markers, , characterized by that the first data set is adjusted to the first auxiliary data set and the second data set is adjusted to the second auxiliary data set. [2] Method according to claim 1, characterized by that the first data set and the first auxiliary data set of the first body region model are generated with a first digitizing device, and that the second data set and the second auxiliary data set of the second body region model are generated with a second digitizing device. [3] Method according to claim 1 or claim 2, characterized bythat a reference system (5) for the first auxiliary data set and for the second auxiliary data set is determined on the basis of a plurality of first and second auxiliary reference markers which are spaced apart from one another and are not arranged rotationally symmetrically. [4] Method according to one of the preceding claims, characterized by that the reference marker device (4) has an impression layer (13) with a pasty, hardening impression material and is molded onto the transition region of the body region to be imaged with the first and second body region models before hardening, and that the first auxiliary data set and the second auxiliary data set are generated after the molded impression material has hardened. [5] Method according to one of the preceding claims 1 to 3, characterized bythat the reference marker device (14) has a molding layer (16) with a molding material that can be plastically deformed by a deformation pressure, which is deformed by pressure and molded onto the transition region of the body region to be imaged with the first and second body region models, and that the first auxiliary data set and the second auxiliary data set are generated without pressure application above the deformation pressure. [6] Use of a reference marker device (4, 14) with a carrier element (9, 15) and with a molding layer (13, 16) arranged on the carrier element (9, 15) with a deformable molding material, wherein at least one photogrammetric or surface-profiled reference marking (12, 17) is arranged on the carrier element (9, 15), for carrying out the method according to one of claims 1 to 5. [7] Use of the reference marker device (4, 14) according to claim 6, characterized bythat the reference marker device (4, 14) has a reference plate (11) on which the at least one reference marking (12, 17) is arranged. [8] Use of the reference marker device (4, 14) according to one of claims 6 or 7, characterized by that the reference marker device (4, 14) has at least three symmetrical reference markings (12, 17) which are arranged asymmetrically. [9] Use of the reference marker device (4) according to claim 8, characterized by that the reference mark (12) is a dot or a ring. [10] Use of the reference marker device (4) according to one of claims 6 to 9, characterized by that the reference marking (12) has a high contrast range. [11] Use of the reference marker device (14) according to one of claims 6 to 8, characterized by that the reference marker device (17) has a plurality of projecting reference formations arranged at a distance from one another.

Images