Method and device for determining 3D coordinates of at least one predetermined point of an object

Description

[0001] The present invention relates to a method and a device for determining 3D coordinates of at least one predetermined point of an object and to a computer program product for carrying out the method.

[0002] The introduction of individually optimized lenses makes it possible to address the needs of people with visual impairments and, for example, provide lenses with individually optimized viewing areas. Custom-fitted lenses enable optimal correction of a user's optical visual impairments. Individual calculation and adjustment of lenses is also possible for sports glasses, which are characterized by large curvatures, frame lens angles, and pre-tilt angles.

[0003] In order to fully exploit the optical advantages of customized ophthalmic lenses, especially individually fitted progressive lenses, it is necessary to calculate and manufacture these lenses with knowledge of the user's wearing position and to wear them according to the wearing position used for calculation and manufacture. The wearing position depends on a variety of optical parameters, for example the user's pupil distance, the frame lens angle, the lens tilt of the frame, the corneal vertex distance of the spectacle-eye system and the grinding height of the lenses. These and other parameters, which can be used or are necessary to describe the wearing position, are contained in and can be found in relevant standards such as DIN EN ISO 1366, DIN 58 208, DIN EN ISO 8624 and DIN 5340.Furthermore, it is necessary that the lenses are arranged or centered in a spectacle frame according to the optical parameters used for production, so that the lenses are actually worn in the position of use according to the optical parameters.

[0004] To determine the position of a lens in front of the eye, several optical parameters must be determined. This provides the information needed for grinding and fitting the lens into the frame, as well as allowing optimizations to be made within the lens itself to adapt it to the wearer's intended position.

[0005] For determining such optical parameters, a device with two image recording devices is known from DE 10 2005 003 699 A1, for example. Each of these devices captures an image of a user wearing glasses from different recording directions. From these images, it calculates, for example, three-dimensional (abbreviated: 3D) coordinates of predetermined points of the user wearing glasses. The optical parameters can be determined from these 3D coordinates of the predetermined points.

[0006] Document US 2015 / 0204663 A1 discloses a 3D scanner in which a sequence of light patterns with different light distributions is projected onto an object. Object points at transitions between bright and dark areas of the light pattern are detected.

[0007] Document US 2012 / 0133954 A1 relates to a device and a method for projecting light patterns onto an object. A second light pattern has smaller distances between bright and dark areas than a first light pattern. A third light pattern is complementary to the second light pattern. An image is captured of each light pattern projected onto the object. The object positions at the transitions from bright to dark in the first light pattern are determined using the three images.

[0008] Other devices use two-dimensional calculation methods that determine the desired optical parameters based on multiple images. There are also manual determination options, such as a pupillometer and a pupillary distance ruler.

[0009] These measurement methods depend on the person performing the measurement and the method of performing the measurement. For example, a two-dimensional image cannot accurately determine the individual corneal vertex distance when determining the pupillary distance, resulting in a systematic deviation in pupillary distance that depends on the subject and the frame.

[0010] When using a stereoscopic camera system to determine a user's optical parameters from the 3D coordinates of predetermined points, a correspondence problem arises. The correspondence problem involves identifying corresponding points in two images taken from different perspectives. Only after corresponding points have been determined in both images can a 3D reconstruction of the recorded points be performed.

[0011] In practice, the corresponding points are determined by manually evaluating the images. This manual evaluation is time-consuming and, due to its user dependency, is a potential source of error in the 3D reconstruction.

[0012] The invention is based on the object of providing an improved possibility for determining 3D coordinates of predetermined points of an object, in particular of optical parameters of a user.

[0013] This object is solved by the subject matter of the independent claims, which also define the invention.

[0014] A first aspect relates to a method for determining 3D coordinates of at least one predetermined point of an object, wherein: a) the object is arranged in a measuring area, b) a variable illumination source projects a variable pattern onto the object arranged in the measuring area, c) an image recording device, which is arranged in a known relationship to the illumination source, records an image of at least a partial area of ​​the object illuminated by the variable illumination source, d) the at least one predetermined point is detected in the recorded image, and e) the 3D coordinates of the at least one predetermined point are determined from the recorded image, taking into account the known relationship of the image recording device to the illumination source, if an examination of the recorded image shows that the at least one predetermined point in the recorded image is marked by a feature of the variable pattern.

[0015] The objective of the method can be to determine 3D coordinates of at least one predetermined point of the object. The 3D coordinates can be determined in any 3D coordinate system, e.g., in the Earth's reference system, the object's reference system, the reference system of the device used to capture the image, and / or the image capture device's reference system. The 3D coordinates can subsequently be further processed.

[0016] According to the invention, the object is the head of a human as a user with a pair of spectacle frames attached. Here, the at least one predetermined point is a point of the system comprising the user's head with spectacle frames that can be used to calculate optical parameters, e.g., a pupil center. The at least one predetermined point can depend on the method. The method can also be designed and provided to determine the respective associated 3D coordinates for a plurality of predetermined points.

[0017] The method is described in connection with determining optical parameters from the determined 3D coordinates. In principle, the method can also be used in other application areas, e.g. in the dental and / or orthopedic field, where model-based surfaces and / or bodies are used, for which a defined number of parameters that can be determined based on the 3D coordinates of predetermined points are used for description. Furthermore, the method can also be used to determine 3D coordinates in industrial manufacturing processes that are controlled mechanically, e.g. by a robot. This can be used, for example, in the assembly of circuit boards, where predetermined points (such as drill holes) on the circuit boards to be assembled are located by the method. Furthermore, e.g.Components with defined test points are used as objects, where the test points are, for example, castings or workpieces with attachment points, such as drill holes. Generally, the at least one predetermined point is a well-defined, predetermined point on the object.

[0018] Both the image capture device and the illumination source are aligned with the measurement area. The measurement area can be an extended three-dimensional or two-dimensional area, or it can be defined by a single point, such as the viewpoint of a user wearing eyeglass frames.

[0019] The variable illumination source is designed and intended to project a variable pattern onto the object arranged in the measurement area. The term "variable" means that the illumination source can project not just a single pattern, but a plurality of different patterns, and / or that the projection direction in which the illumination source projects the pattern is variable and thus changeable.

[0020] The illumination source can have individual pixels that can be specifically controlled. For example, a screen, such as an LCD, can be used as the illumination source. Depending on which pixels of the illumination source are controlled, different patterns can be emitted from the illumination source.

[0021] For the process, it may be sufficient if the variable illumination source has a number of pixels that at least corresponds to the VGA standard, e.g., at least 320 x 200 pixels or at least 640 x 480 pixels. The illumination source is controllable and can be controlled during the process so that the pattern emitted by the illumination source is varied. The illumination source can also have one or more optical elements for influencing the emitted pattern, such as deflecting mirrors, prisms, lenses, polarization beam splitters, λ / 4 plates, λ / 2 plates, etc.

[0022] A camera, in particular a digital camera, can be used as the image recording device, for example, which can record a digital image of at least a partial area of ​​the object. The image recording device can further comprise at least one optical deflection element, such as a deflection mirror, etc. Image data of the partial area of ​​the object can be deflected via the deflection element.

[0023] The image recording device records an image of at least a partial area of ​​the object illuminated by the illumination source, wherein the image recording device can generate image data. The image data can be in the form of digital data of the recorded image. The recorded image and thus also the recorded image data contain at least a partial area of ​​the object that is illuminated by the pattern of the illumination source. A single image recording device, which is statically fixed, for example, may be sufficient to carry out the method. The method therefore does not use a stereoscopic camera system and / or does not require stereoscopic image analysis. The 3D coordinates can be determined from the image data of a single recorded image.

[0024] The image recording device is arranged in a known relationship to the illumination source. This means that an optical axis of the image recording device, which runs, for example, through the center of a lens of the image recording device, is arranged in a fixed relationship relative to an illumination direction of the illumination source. Since both the optical axis of the image recording device and the illumination direction of the illumination source can be deflected by deflecting elements, within the scope of the invention, in particular the section of the optical axis of the image recording device in relation to the section of the illumination direction that ultimately strikes or intersects the object arranged in the measuring area is known in advance.

[0025] The previously known relation may include an angle between the optical axis and the illumination direction as well as a distance of the image recording device from the illumination source.

[0026] The previously known relation can alternatively or additionally comprise associated distance information for each individual pixel of the illumination source, for example in the form of a table, assignment function, and / or list. The previously known relation can thus, in particular, include an adjustment and / or calibration of the image recording device relative to the illumination source. Furthermore, the previously known relation can include a distance of the image recording device from the illumination source or a distance of a deflection element of the image recording device from a deflection element of the illumination source. In addition to the distance, 3D coordinates of the aforementioned locations can, for example, be previously known as part of the previously known relation.

[0027] According to the method, at least one predetermined point is detected in the image captured by the image capture device. Detection can be performed manually or semi-automatically with software support. In particular, detection can also be fully automated, e.g., with the aid of suitable software programs. In this detection step, the at least one point is detected in the captured image data; preferably, all of the predetermined points are detected. Optionally, the detected point(s) can be marked. An operator performing the method or monitoring the method's implementation can then check the marked points and thus the detection step.

[0028] After the detection of at least one predetermined point, the recorded image is checked. This check can be carried out either automatically, e.g. entirely by software, manually, e.g. by the aforementioned operator, e.g. an optician, or semi-automatically, e.g. by the operator who receives a suggestion determined by software. When checking the recorded image, a check is carried out to determine whether the at least one predetermined point is marked by the feature of the variable pattern. The feature of the variable pattern can be, e.g., a line, a contrast edge, a cross, and / or another feature of the variable pattern. Preferably, the feature is in particular a thin line of light, e.g., only one pixel wide, or a contrast edge, e.g., a sharp transition from an area illuminated in the pattern to an area not illuminated by the pattern.Alternatively or additionally, the feature may also be a transition from a first color to a second color of the pattern, and / or a transition from a first brightness level of the pattern to a second brightness level of the pattern.

[0029] A check is carried out to determine whether at least one predetermined point in the recorded image is marked by such a feature. This check is then affirmed or assessed positively if the feature of the pattern, e.g. the line or the contrast edge, runs through the predetermined point. A tolerance of e.g. up to five pixels, preferably up to one pixel, between the predetermined point and the feature in the recorded image can be taken into account. The marking can be detected relatively accurately in particular if an elongated feature of the pattern, e.g. a line or a contrast edge, intersects with an elongated characteristic of the object, such as a section of a pair of spectacle frames. Such an intersection point between the feature of the pattern and an elongated characteristic of the object at an angle can be evaluated and / or assessed during the check as a marking of the predetermined point.

[0030] From the previously known relationship between the illumination source and the image recording device, the coordinates in three-dimensional space assigned to the predetermined point marked in the image data can thus be determined, ascertained and / or calculated using the image data. The 3D coordinates can thus be determined, for example, by triangulation. Alternatively or additionally, the associated 3D coordinates can be determined from a previously known relationship between the individual pixels of the illumination source in the recorded image and a distance. In this way, when adjusting a permanently installed, static illumination source and a permanently installed, static image recording device, a fixed relationship between each individual pixel of the illumination source and the distance from the image recording device can be measured and saved. Such an adjustment is explained in more detail below.

[0031] The 3D coordinates can be calculated, for example, as Cartesian coordinates, as cylindrical and / or spherical coordinates. The 3D coordinates do not necessarily have to be output at the end of the process. This allows the 3D coordinates to be further processed, particularly for calculating a user's optical parameters. These optical parameters, and not the 3D coordinates determined for their calculation, can be output. In other applications, for example, only a pass / fail statement based on the determined 3D coordinates can be output.

[0032] With this method, only the coordinates of one of the predetermined points can be determined from an image, and the coordinates of the remaining predetermined points from other, specifically assigned images, which are taken sequentially, for example. Alternatively, all 3D coordinates of all predetermined points can be determined simultaneously from a single image. To do this, the verification checks whether not just the one predetermined point, but all of the predetermined points are marked by at least one feature of the variable pattern. Only then are the corresponding 3D coordinates determined.

[0033] The method is not a stereoscopic method. This means that determining the 3D coordinates does not require solving a correspondence problem, which, as is common in stereoscopic methods, requires finding corresponding points in images taken from different angles. Solving the correspondence problem is eliminated in the method according to the first aspect. This reduces potential sources of error.

[0034] At the same time, it is possible to perform the method fully automatically or at least semi-automatically, i.e., with the aid of a processor and software running on the processor, e.g., a computer program product. Both the detection of the predetermined points in the captured image and the verification of whether these points are marked by a feature of the pattern can be performed fully automatically or at least semi-automatically. This facilitates and simplifies the implementation of the method.

[0035] Furthermore, the method can be implemented using a single image recording device, which can be statically installed and designed and configured to capture an image of the object arranged in the measurement area from only a single image recording direction along its optical axis. Thus, the method can, in particular, dispense with the need for an additional image recording device and a stereoscopic camera system.

[0036] According to the invention, if the review of the recorded image reveals that the at least one predetermined point in the recorded image is not marked by a feature of the variable pattern, the variable pattern projected onto the object by the illumination source is varied, and steps c) to e) are then repeated under illumination with the varied pattern. If the review thus reveals that either none of the predetermined points, or not all of the predetermined points, in the image data, i.e. the recorded image, are not marked by at least one feature of the variable pattern, the pattern is varied. The pattern can be varied, for example, by controlling at least one subgroup of the pixels of the variable illumination source differently than when the previous image was recorded. This makes it possible to provide a varied pattern.When varying the pattern, in particular the features of the variable pattern can be shifted (e.g. laterally in the two-dimensional image), for example the lines, crosses, and / or contrast edges of the pattern. The variation of the pattern causes the features of the pattern to be arranged at different locations when the next image is taken, for example at the predetermined points. When varying the pattern, it can be taken into account in particular that one or some of the predetermined points are already marked by a feature of the variable pattern. These sub-regions of the variable pattern can be kept constant, i.e. not varied. In this case, for example, the control of a subgroup of the pixels of the illumination source that are responsible for forming the sub-region with the feature that marks one of the predetermined points can be left constant.To carry out the method, it is sufficient to vary the pattern only in those partial areas of the recorded image (e.g. by controlling a corresponding subgroup of pixels) in which one of the predetermined points is arranged which is not yet marked by a feature of the variable pattern.

[0037] After the pattern has been varied, the image recording device, as described in step c), records an image of the partial area of ​​the object illuminated by the variable illumination source. The object is illuminated by the changed pattern, i.e., the varied pattern. Subsequently, the predetermined point(s) are detected in the newly recorded image, and it is again checked whether the predetermined point(s) are each marked by at least one feature of the varied pattern. Even if several images are recorded consecutively in this method, only a single one of the recorded images is used to calculate the 3D coordinates of the at least one predetermined point, in particular the one in which the predetermined point is marked by the feature of the variable pattern.

[0038] According to a further development of this embodiment, the variable pattern projected onto the object by the illumination source is varied and then steps c) to e) of the method are repeated with the varied pattern until the at least one predetermined point in the recorded image is marked by a feature of the variable pattern and the 3D coordinates of the at least one predetermined point are calculated. The method can therefore be iterated and / or repeated until the 3D coordinates of the one predetermined point or the several predetermined points can be determined. As long as the at least one point in the last recorded image is not yet sufficiently marked by a feature of the variable pattern, the pattern projected into the measuring area is changed and method steps c), d) and e) are repeated.This iterative process ensures that the 3D coordinates of at least one predetermined point are first specifically marked and can then be reliably determined.

[0039] According to one embodiment, the variable pattern is deliberately varied such that the varied pattern is expected to mark the at least one predetermined point of the object with a feature of the varied pattern. In other words, when varying the projected pattern, consideration is given to where in the last recorded image the at least one predetermined point was detected. However, since the 3D coordinates of the predetermined point are not yet known, and cannot be determined from the two-dimensional image without the marking, the two-dimensional coordinates of the point in the last recorded image can only be used to vary the pattern such that a feature of the pattern is arranged at a probable, estimated location of the predetermined point.This estimate of the approximate location of the predetermined point can be improved by an iterative approximation, i.e., a repetition of steps c) to e) of the method while varying the pattern, such that a feature of the pattern iteratively approaches the predetermined point. The variation of the pattern thus takes into account an estimate of the location of the predetermined point from the previously acquired two-dimensional image, and can also be improved by accessing previously known inventory data. Taking into account the estimate of a probable location of at least one predetermined point reduces the number of iteration steps required to calculate the actual 3D coordinates and thus shortens the time required to carry out the method.

[0040] According to one embodiment, successive images are recorded at a repetition rate of at least 10 Hz until the 3D coordinates of the at least one predetermined point are calculated. The step of detecting, checking, and varying the pattern follows automatically and so quickly that the next image with a varied pattern can be recorded after just one tenth of a second. The image repetition rate is thus at least 10 Hz, preferably at least 20 Hz. The high image processing speed, variation speed, and recording speed can ensure that, in the event of slight movement of the object, such as a user wearing spectacles, the pattern can be successfully varied such that the predetermined points of the object are marked in the recorded image.The high repetition frequency also makes it possible to determine 3D coordinates of predetermined points of an object that is moving at least slowly.

[0041] According to one embodiment, the variable pattern has at least one substantially horizontal and / or vertical contrast edge as a feature, which is shifted and / or changed when the pattern is varied. The contrast edge can in particular be formed as a line, in particular a line only a few pixels wide, e.g., a single pixel. The contrast edge can also be formed as a sharp contrast transition along a line, e.g., between an illuminated area and an unilluminated area of ​​the pattern. The terms "horizontal" and "vertical" refer either to the Earth's reference system or to the object's reference system. In particular, the terms can refer to a vertical and horizontal direction in box dimensions.

[0042] For the purposes of this invention, dimensioning in box dimensions refers to the measurement system as described in relevant standards, for example, DIN EN ISO 8624 and / or DIN EN ISO 1366 and / or DIN 58208 and / or DIN 5340. Furthermore, with regard to box dimensions and other conventional terms and parameters used, reference is made to the book "Die Optik des Auges und der Sehhilfen" (The Optics of the Eye and Visual Aids) by Dr. Roland Enders, 1995, Optische Fachveröffentlichung GmbH, Heidelberg, as well as the book "Optik und Technik der Brille" (Optics and Technology of Glasses) by Heinz Diepes and Ralf Blendowski, 2002, Verlag Optische Fachveröffentlichungen GmbH, Heidelberg. The standards and the aforementioned book constitute an integral part of the disclosure of the present application for the definitions of terms.

[0043] In particular, the contrast edge can be shifted essentially perpendicular to the direction of the contrast edge. Thus, essentially horizontal contrast edges can be shifted in an essentially vertical direction, while essentially vertical contrast edges can be shifted in an essentially horizontal direction. The actual orientation of the respective contrast edge remains intact and / or constant.

[0044] According to one embodiment, the variable pattern has a number of geometric shapes with contrasting edges as features. When the pattern is varied, the number of geometric shapes and thus the number of contrasting edges of the pattern is increased. In particular, the number of geometric shapes can be increased in a partial area of ​​the pattern in which the at least one predetermined point has been detected in the recorded image. Rectangles, triangles, and / or trapezoids can be used as geometric shapes. As a simple example, a geometric shape such as a rectangle can be determined in the last recorded image in which the predetermined point is located. In the subsequent image recording, the pattern is varied such that at least the geometric shape in which the point was detected is further subdivided. For example,a rectangle as a geometric shape can be divided into smaller, essentially equal-sized geometric shapes, such as several smaller rectangles. A rectangle could, for example, be divided into four smaller rectangles that are essentially the same size. Alternatively, the rectangle could also be divided into several rectangles, e.g. 16 or more. Using several smaller geometric shapes increases the number of contrasting edges in an area of ​​the pattern that is likely to contain at least one predetermined point. If the method is to determine the 3D coordinates of several predetermined points, all those geometric shapes that contain one of the predetermined points can be further subdivided when the pattern is varied. This results in a particularly efficient variation that can lead to successful determination of the 3D coordinates after as few iteration steps as possible.

[0045] According to the invention, the object is a user's head with a pair of spectacle frames arranged therein in the wear position. The partial region of the user's head with the spectacle frame arranged thereon in the wear position, which is recorded by the image recording device, can comprise at least one pupil, both pupils, and / or the pupils surrounded by the spectacle frame. The wear position is defined, for example, in the standards described above. In this embodiment, the method can be used to calculate the user's optical parameters, which can be taken into account, for example, when providing customized spectacle lenses.

[0046] In a further development of this embodiment, at least one of the following points is used as the at least one predetermined point: a pupil center, an internal temporal frame point, an internal nasal frame point, an internal frame point above the pupil and / or an internal frame point below the pupil.

[0047] Preferably, the method determines the 3D coordinates of all five aforementioned points for each of the user's two eyes. Thus, the method can determine a total of the ten aforementioned points. From the 3D coordinates of these ten points, most of the user's relevant optical parameters can be determined. Alternatively or additionally, additional frame boundary points, particularly in the box dimension, can be determined.

[0048] Here, the user's optical parameters can be determined from the specific 3D coordinates of the predetermined points. In particular, the pupil distance can be calculated as an optical parameter as the length between the pupil centers in three-dimensional space. In addition, the pupil distance can be divided into a right pupil distance and a left pupil distance as additional optical parameters. Furthermore, the optical parameter frame lens angle can be calculated for a left and / or right side of the spectacle frame. Furthermore, the optical parameters lens length and / or lens height, for example, can be determined, which can only be calculated approximately using the ten points mentioned. For a more precise calculation of these parameters, the box dimension listed above can be used, which can be taken into account in the calculation.

[0049] According to a further development of the method, in which a user with a spectacle frame arranged in the wear position is used as the object, the 3D coordinates of a pupil center of the user are determined by marking at least two points adjacent to the pupil center as predetermined points, each with at least one feature of the variable pattern, from whose 3D coordinates the 3D coordinates of the pupil center are estimated. The problem with the pupil center is that the pupil center itself can only be marked to a limited extent by an illumination source. Thus, the pupil center is biologically designed to guide the incoming light into the interior of the eye. This can therefore result in distortion in the image data at this point.Instead of directly marking the respective pupil center with a feature of the pattern, not the actual pupil center but two, three, or more points adjacent to the pupil center are specifically selected as predetermined points and marked by the pattern. The actual 3D coordinates of the pupil center can be determined from an average of the neighboring points. For example, a point on the iris located horizontally to the right and left of the pupil center can be marked, and its 3D coordinates can be determined. This simplifies and improves the determination of the 3D coordinates of the right and / or left pupil center.

[0050] A second aspect relates to a device for determining 3D coordinates of at least one predetermined point of an object. The device comprises a variable illumination source for projecting a variable pattern, the object arranged in a measurement area, and an image recording device arranged in a known relationship to the illumination source and configured and provided to record an image of at least a partial area of ​​the object illuminated by the variable illumination source.The device comprises a detection module for detecting the at least one predetermined point in the recorded image and a coordinate determination module for determining the 3D coordinates of the at least one predetermined point from the recorded image, taking into account the previously known relationship of the image recording device to the illumination source, if a review of the recorded image reveals that the at least one predetermined point in the recorded image is marked by a feature of the variable pattern.

[0051] The device can be used in particular for carrying out the method according to the invention. Therefore, all statements made regarding the method also apply to the device, and vice versa. The device can be designed as a static device that is arranged in a fixed location, e.g., at a predetermined location in a room. The measuring area can be an area spaced apart from the device in the same room. Both the illumination source and the image recording device are statically fixed to the device, resulting in the known relationship. Furthermore, the relationship between the measuring area and the device can also be known in advance.

[0052] Both the detection module and the coordinate determination module can comprise a graphical evaluation of image data from the recorded image. Both the detection module and the coordinate determination module can be implemented as a software-supported module. The verification of the recorded image for a marking of the at least one predetermined point can also be performed with software support, e.g., by a corresponding verification module. The individual modules of the device can be implemented, for example, on a processor, in particular on the same processor.

[0053] According to the invention, the device additionally comprises a pattern variation module which, if the review of the recorded image reveals that the at least one predetermined point in the recorded image is not marked by a feature of the variable pattern, varies the pattern projected onto the object by the illumination source. The pattern variation module can carry out variation options as described above in connection with the method, in particular changing and / or shifting contrast edges of the pattern, and / or changing the number and size of geometric shapes in the pattern, in particular in the region of the pattern in which the at least one predetermined point is arranged. The pattern variation module can vary the pattern for as long and as often as necessary until an image subsequently recorded by the image recording device can be used to determine the 3D coordinates.This means that the pattern is varied until the last image taken is checked to see that at least one predetermined point is marked by at least one feature of the variable pattern.

[0054] According to a further development, the device has a parameter determination device which is designed to determine optical parameters of a user from the determined 3D coordinates, provided that the head of the user with a spectacle frame arranged therein in the position of use is used as the object.

[0055] The invention relates to a computer program product comprising program parts which, when loaded into a computer, are designed to carry out and / or control a method according to one of claims 1 to 9. The computer program product can in particular be a software product and / or a device with a computer processor onto which corresponding software can be loaded.

[0056] The invention is explained in more detail below with reference to exemplary embodiments illustrated in the figures. They show: Fig. 1A is a schematically illustrated top view of a user with elements of a device for determining 3D coordinates of predetermined points; Fig. 1B is a schematically illustrated side view of the user with elements of the device for determining 3D coordinates of predetermined points; Fig. 2 is a schematic illustration of image data of an image of a user with spectacle frames, recorded by an image recording device; Figs. 3A, 3B, and 3C are a schematic illustration of patterns projected by an illumination source; Fig. 4 is a schematic illustration of a partial area of ​​an image of a user with spectacle frames, recorded by an image recording device; Fig. 5A is a schematic illustration of a first pattern projected by an illumination source; Fig. 5B is a schematic illustration of a second pattern resulting from a variation of the Fig. 5Ashown first pattern; Fig. 5C in a schematic representation a third pattern which is generated from a variation of the in Fig. 5B shown second pattern; Fig. 5D in a schematic representation a fourth pattern which is generated from a variation of the in Fig. 5C shown third pattern is generated; Fig. 6A in a schematic representation a first image of a user's eye surrounded by a spectacle frame, onto which the Fig. 5A shown first pattern is projected; Fig. 6B in a schematic representation a second image of a user's eye framed by a spectacle frame, onto which the Fig. 5B shown second pattern is projected; Fig. 6C in a schematic representation a third image of a user's eye framed by a spectacle frame, onto which the Fig. 5Cshown third pattern is projected; Fig. 6D in a schematic representation a fourth image of a user's eye framed by a spectacle frame, onto which the Fig. 5D shown fourth pattern is projected; and Fig. 7 is a flowchart of a method for determining 3D coordinates of at least one predetermined point.

[0057] Figures 1A and 1B show a schematic representation of a user 100 in the measuring range of a device for determining 3D coordinates of predetermined points. Only an image recording device 10 and an illumination source 20 are shown as elements of the device. Both the image recording device 10 and the illumination source 20 are aligned with a measuring range. The measuring range is shown in the Figures 1A and 1Bnot specifically marked. The user 100 is arranged in the measuring range, more precisely the head of the user 100 with a spectacle frame 101 arranged thereon in the use position. In the terminology used above, the head of the user 100 and the spectacle frame 101 arranged thereon in the use position form an object 105 that is arranged in the measuring range of the device. The use position is defined in the standards outlined above. The device is designed and provided to detect predetermined points on the object 105 and to calculate their 3D coordinates. This serves to calculate optical parameters such as a pupil distance, a frame lens angle, a spectacle lens inclination, a corneal vertex distance between the spectacles and the eye, and a grinding height of the spectacle lenses as optical parameters.

[0058] The image recording device 10 can be designed as a digital camera and has an optical axis 11. The optical axis 11 is directed from the center of a recording lens and / or aperture of the image recording device 10 onto the measuring area and defines a recording direction of an image that can be recorded by the image recording device 10. Similarly, the illumination source 20 is directed onto the measuring area such that an illumination direction 21 from the illumination source 20 is directed onto the measuring area. The illumination direction 21 defines a direction in which light is emitted from the illumination source 20 onto the measuring area. The illumination source 20 is designed and provided to emit light in the form of a structured pattern. The pattern emitted by the illumination source 20 is thus designed as a structured light pattern.This structured light pattern is projected onto the object 105, in the exemplary embodiment shown onto the head of the user 100 and the spectacle frame 101 arranged thereon. The structured light pattern can in particular be designed as a substantially monochrome light pattern, that is to say, for example, formed from light of a single wavelength. The light pattern has bright and dark areas, that is to say partial areas of the light pattern illuminated by the light and partial areas of the light pattern not illuminated by the light. The light pattern can have features with a contrast transition, in particular contrast edges, as well-defined, at least partially rectilinearly arranged transitions from a partial area of ​​the pattern with a high light intensity to a partial area of ​​the pattern with a low light intensity and vice versa. In this case, a partial area of ​​low light intensity can, for example,be formed as a partial region of the light pattern whose light intensity is at least 50% lower, preferably at least 75% lower, particularly preferably at least 95% lower than the light intensity of the adjacent partial region with the high light intensity. In one embodiment, the at least one partial region with low and / or lower light intensity is not illuminated by the illumination source.

[0059] The pattern projected by the illumination source 20 into the measuring area is variable. This means that the illumination source 20 can emit different patterns. For this purpose, the illumination source 20 can be designed, for example, as a screen with individual pixels that can be controlled individually. A variation in the pattern emitted by the illumination source 20 can thus be based on a variation in the light emitted by each individual pixel of the illumination source 20. The illumination source can thus vary the pattern it emits on a pixel-by-pixel basis. Subgroups of pixels can be assigned to partial regions of the pattern in such a way that the light intensity of one or more partial regions of the pattern is controlled and / or regulated by controlling one or more subgroups of pixels.

[0060] The image recording device 10 and the illumination source 20 are arranged in a known relationship to one another. This may mean that the image recording device has a known distance in three-dimensional space from the illumination source 20. Furthermore, the arrangement and in particular the direction of the optical axis 11 may be known in advance, as may the arrangement and direction of the illumination direction 21. In particular, an angle between the optical axis 11 and the illumination direction 21 may be known in advance.

[0061] In the Figures 1A and 1BIn the embodiment shown, the optical axis 11 and the direction of illumination 21 intersect at an intersection point 30. This intersection point 30 is located in the measuring range of the device. In the exemplary embodiment shown, the intersection point is located approximately between the user's eyes on the bridge of the nose. In alternative embodiments, the optical axis 11 and the direction of illumination 21 do not necessarily have to intersect at a point, but can simply be at a minimum distance from one another in the measuring range. Such a minimum distance from one another can, for example, be a maximum of 10 cm. From the previously known arrangements of the image recording device 10 and the illumination source 20 as well as the associated optical axis 11 and the direction of illumination 21, the 3D coordinates of predetermined points in the image recorded by the image recording device 10 can be calculated by means of triangulation.

[0062] Alternatively or additionally, the device can be adjusted, calibrated, and / or fixed with the image recording device 10 statically fixed to the illumination source 20. A calibration object can be used to calibrate the device, i.e., to record and / or store the previously known relationship between the illumination source 20 and the image recording device 10. For the adjustment, a formula and / or a correspondence table can be created for each individual pixel of the illumination source 20, which assigns a 3D coordinate to each individual pixel of the illumination source 20 depending on the location of this pixel in the recorded, two-dimensional image of the image recording device.Since the illumination source 20 and the image recording device 10 are arranged relative to one another in such a way that the optical axis 11 and the illumination direction 21 either intersect at an angle, or the projections of these two straight lines onto a horizontal and / or vertical plane intersect at an angle, i.e. these two straight lines are not aligned parallel to one another, a relationship as mentioned above can be determined, calculated and / or stored for each pixel of the illumination source.

[0063] Thus, each 2D coordinate of each pixel in the recorded image, e.g., each XY coordinate in the image recorded by the image recording device 10, can be assigned a 3D coordinate in the measurement range. In general, the 2D coordinates in the recorded image depend on the position of the respective illuminated point of the object relative to the position of the image recording device 10 and the illumination source 20. Such a relationship can be predetermined and / or stored at least for those object points that are located in the measurement range. Mathematically expressed, an assignment function can be defined as a previously known relation between the image recording device 10 and the illumination source 20 for each pixel p of the illumination source. fp pre-stored in the following form: f p x y ↦ x ′ , y ′ , z ′ .

[0064] Here the coordinates are (x, y) 2D coordinates of the point in the captured image illuminated by pixel p. The 2D coordinates (x, y)can also be the coordinates of a center point of an area illuminated by the pixel p in the captured image. The coordinates (x', y', z ') are 3D coordinates of the location and / or point in the measurement area that is illuminated by the pixel p and that causes the illuminated point and / or area in the recorded image.

[0065] The assignment function fp can be pre-stored for each pixel p or a plurality of pixels p of the illumination source 20 in the form of a mathematical function and / or in the form of an assignment table in a memory module of the device.

[0066] Thus, by determining the pixel of the illumination source 20 that illuminates a detected one of the predetermined points of the object 105, the 3D coordinate of the thus illuminated and / or marked point of the object 105 can be determined. In other words, when carrying out the method, it can be determined, for example, which pixel of the illumination source 20 illuminates the predetermined point detected in the image. From this, the associated 3D coordinate can be calculated either by triangulation or via the assignment function described above. The 3D coordinates can be determined automatically from the image data that are assigned to and / or correspond to the image captured by the image recording device 10.

[0067] Fig. 2shows a schematic representation of an image that has been recorded and generated by the image recording device 10. In the exemplary embodiment shown, the image data of a schematically represented frontal image of a portion of the head of a user 100 with the spectacle frame arranged in the use position are shown. Figure 2 only two spectacle lenses 110, a spectacle frame 101, a right eye 112 and a left eye 114 of the user 100 are shown. As predetermined points in Fig. 2 a right pupil center RPM of the right eye 112 and a left pupil center LPM of the left eye 114 are shown. Furthermore, Fig. 2a boundary 120 of the right lens edge or frame edge 102 around the right eye 112 and a boundary 122 of the left lens edge or frame edge 102 around the left eye 114 in the box dimension. Furthermore, a nasal right frame point NRF and a temporal right frame point TRF of a horizontal plane HR in the user's reference system 100 with the frame edge 102 relative to the right eye 112 are shown. Fig. 2 Also shown are an upper right frame point ORF and a lower right frame point URF of a vertical plane VR in the reference system of the user 100 perpendicular to the horizontal plane HR of the user 100 with the spectacle frame edge 102 with respect to the right eye 112.

[0068] Analogously, a nasal left frame point NLF and a temporal left frame point TLF are shown in a horizontal plane HL in the reference system of the user 100 with the spectacle frame edge 102 relative to the left eye 114. In Fig. 2 Also shown is an upper left frame point OLF and a lower left frame point ULF of a vertical plane VL in the reference system of the user 100 perpendicular to the horizontal plane HL of the user 100 with the spectacle frame rim 102 with respect to the left eye 114.

[0069] Figure 2shows points of the system consisting of the user's head 100 and the spectacle frame 101 arranged thereon in the wear position. To determine optical parameters that can be used for the production and / or calculation of individual spectacle lenses, in particular individual ophthalmic spectacle lenses, the 3D coordinates of predetermined points are determined in this system. According to one embodiment, the 3D coordinates of the following ten points are determined: right pupil center RPM, left pupil center LPM, temporal right frame point TRF and temporal left frame point TLF, nasal right frame point NRF and nasal left frame point NLF, upper right frame point ORF and upper left frame point OLF, and lower right frame point URF and lower left frame point ULF.

[0070] The most important optical parameters can be determined from the 3D coordinates of the aforementioned ten points as predetermined points.

[0071] In this method, the predetermined points are specifically marked by the pattern projected by the illumination source 20. In particular, the eight frame points, namely the ORF, NRF, URF, TRF, OLF, TLF, ULF, and NLF points, can be marked by a contrast edge of the pattern intersecting at an angle with the spectacle frame edge (and / or lens edge) 102 in the recorded image. The intersection angle can, in particular, be a substantially right angle and / or an intersection angle between approximately 30° and approximately 150° in order to clearly mark the respective frame point in the image.

[0072] Such a contrast edge of the projected pattern can be provided, for example, by a line that is specifically placed through the respective frame point ORF, NRF, URF, TRF, OLF, TLF, ULF and / or NLF. For example, one line each along the vertical planes VR and VL and one line each through the horizontal planes HR and HL could be projected onto the eye in order to specifically mark the eight frame points. The lines projected onto the object 105 in this way can, for example, be only a single pixel wide in order to enable the most precise and targeted marking of the predetermined points. However, a contrast edge between an unlit and an illuminated area of ​​the pattern can also be used as a pattern.

[0073] To detect and mark the two pupil centers LPM and RPM, the corresponding 3D coordinates can be determined indirectly. Since the human pupil is biologically designed to direct a large portion of visible light into the eye rather than reflecting it, technical difficulties can arise when marking the pupil center using a feature of the pattern. Therefore, in some cases, the 3D coordinates of the pupil centers LPM and RPM cannot be determined directly like the frame edge points. Therefore, within the scope of the method, the pupil centers themselves are not directly marked by a feature of the projected pattern, but instead points adjacent to the pupil centers LPM and RPM, e.g., points on the iris of the respective right or left eye 112 and 114.

[0074] In Figure 2Instead of the right pupil center RPM, three iris points are marked with the features R1, R2, and R3 of the pattern, which are arranged adjacent to the right pupil center RPM on the iris of the right eye 112. First, these three iris points can be marked as predetermined points of the object 105 by the features R1, R2, and R3 of the pattern. Subsequently, the 3D coordinates of the three corresponding iris points can be determined as described above. The 3D coordinates of the right pupil center RPM can then be determined from the 3D coordinates of the three neighboring iris points. The 3D coordinates can be determined, for example, as an average of the 3D coordinates of the three iris points, in particular depending on a geometry of the three iris points relative to one another on the respective right eye.

[0075] Analogously, in Figure 2Nor is the left pupil center LPM directly marked by a feature of the pattern, but rather only three points on the iris of the left eye 114 that are adjacent to the left pupil center LPM. These three points are marked by the three features L1, L2, and L3 of the pattern. Their 3D coordinates can be determined as described above. The 3D coordinates of the left pupil center LPM can be determined depending on the 3D coordinates of these three iris points.

[0076] Figures 3A, 3B and 3Cshow three differently projected patterns A, B and C with strongly pronounced contrast edges. The dark partial areas in patterns A, B, C are unlit, while the light partial areas are illuminated by the illumination source 20. The patterns A, B and C can thus be designed as single-color patterns, which, for example, consist exclusively of unlit and illuminated partial areas (e.g., with a single color), which are each delimited from one another by contrast edges. The contrast edges are, at least in sections, essentially straight. Straight sections of the contrast edges form features of the patterns A, B, C that are particularly well suited for marking previously known points.

[0077] The monochromatic light, for example, can be light of a single, specific wavelength, or it can be composed of light of different wavelengths, such as white light. In particular, the light can also be emitted in a wavelength range that is not detected by the user 100 and therefore does not dazzle them, such as the infrared range.

[0078] The pattern is projected into the measurement area in at least one color, with at least one wavelength, and / or in at least one wavelength range that can be recognized and / or detected in the image captured by the image capture device 10, i.e., is detectably contained in the image data. The image capture device 10 used is thus sensitive in the wavelength range used by the illumination source 20.

[0079] Figure 4shows, in a schematic representation, only one half of the image recorded by the image recording device 10. In the image data shown, only the left eye 114 of the user 100 is shown, which is bordered by the left spectacle frame edge 102. Analogously, the entire image can also contain the right pupil and the right eye 112 of the user 100, which is bordered by the right spectacle frame edge 102. In general, the head of the user 100 is preferably arranged in the measuring area such that both eyes 112 and 114 of the user 100 are imaged completely and frontally in the image data produced by the image recording device 10, each of which is completely bordered by the spectacle frame edge or spectacle lens edge 102.

[0080] Figure 4In particular, it shows the left pupil center LPM as the center of four iris points of the left eye 114, which are marked by four features L1, L2, L3, and L4 of the pattern emitted by the illumination source 20. Each of the features L1 to L4 has at least two crossed lines, the intersection of which marks an iris point on the iris of the left eye 114. Using the known relationship between the image recording device 10 and the illumination source 20, the 3D coordinates of the corresponding four points on the iris of the user 100 can be determined. The 3D coordinates of the left pupil center LPM can be determined depending thereon, in particular as an average of the 3D coordinates of the four iris points. In addition, the coordinate(s) indicating the distance from the image recording device 10 can be corrected based on known measurement data.These previously known measurement data can take into account and / or include the fact that the eyeball itself is round and the pupil center is therefore offset outwards relative to the marked iris points, e.g. in the direction of the image recording device 10.

[0081] In general, the 3D coordinates of a pupil center can be determined by first determining the 3D coordinates of at least two neighboring points (e.g., iris points on the iris of the same eye), from which the 3D coordinates of the pupil center are determined. As in Figure 2 This can be done using either three iris points, or as shown in Figure 4 shown using at least four iris points.

[0082] Figure 4further shows a feature M1 of the pattern emitted by the illumination source. The feature M1 is formed as a substantially vertically arranged line of the pattern of the illumination source 20, which is substantially perpendicular to a direction of extension of the spectacle frame edge 102. Since the spectacle frame edge 102 is arranged closer to the image recording device 10 than the face of the user 100 arranged behind it, the feature M1 illuminates the Figure 4The image data shown shows several feature sections M1', M1" and M1'" that are laterally offset from one another. Thus, the two feature sections M1' and M1‴, which illuminate the face of the user 100 in the image, are laterally offset from the feature section M1", which illuminates a section of the spectacle frame rim 102 in the image. All three feature sections M1', M1", and M1"' are shown in the image data as essentially vertical lines. Due to the spatial offset of the spectacle frame rim 102 relative to the face of the user 100 behind it, the feature of the pattern in the image data exhibits two jumps: above the feature section M1' on the face to the feature section M1" on the spectacle frame rim 102, and above it back onto the face, namely to the feature section M1'". As a predetermined point, the coordinates of the lower left frame point ULF can be determined by targeted marking with the feature M1 of the pattern.For this purpose, it can be determined in an intermediate step which pixels of the illumination source 20 illuminate the feature section M1" and which assignment function . fp assigned to these pixels.

[0083] Figure 5A shows a first pattern 200, which is projected by the illumination source 20 into the measurement area, in particular along the illumination direction 21. The first pattern 200 has only a substantially rectangular illuminated partial area and a likewise substantially rectangular unilluminated partial area. A feature M1 of the first pattern 200 is arranged between the illuminated partial area and the unilluminated partial area of ​​the first pattern 200. The feature M1 is designed as a contrast edge, in particular as a contrast edge that is at least partially rectilinear. A predetermined point of the object 105 can be marked with the rectilinear section of the contrast edge of the feature M1.

[0084] Figure 6A shows a schematic representation of a first image 300 recorded by the image recording device 10. This first image contains only one eye of the user 100, namely the left eye 114. Alternatively, the image could also contain both of the user's eyes, each framed by the spectacle frame edge (and / or lens edge) 102. The first image 300 recorded in this way shows how the first pattern 200 of the illumination source 20 illuminates a partial area of ​​the face of the user 100 and the spectacle frame 101 arranged thereon in the use position. It should be noted that the Figure 6AThe representation shown is merely schematic. In realistic images, the contrast edge, i.e., feature M1, would no longer be consistently straight, but would depend on the surface structure of the illuminated object, i.e., the face of the user 100 and the spectacle frame edge 102. For reasons of simplicity, however, only this schematic representation is shown in the figures, in which feature M1 continues to be shown as straight.

[0085] When carrying out the method for determining the 3D coordinates of predetermined points, the predetermined points for which the 3D coordinates are to be detected are first determined in the recorded first image 300. Examples of these predetermined points are shown in Figure 6AThe left pupil center LPM and the upper left frame point OLF are shown. The predetermined points can be detected automatically in the image data of the first image 300, in particular with the aid of software. In the image data of the first image 300, the predetermined points can be detected either by image processing and / or by manual selection. Predetermined points can be, in particular, the aforementioned frame points and pupil centers, but also the corners of the eyes, corners of the mouth, etc.

[0086] The predetermined points are shown in subsequent images, for example in the following Figures 6B, 6C and 6D are shown, are detected again. It is advantageous if subsequent images are captured as quickly as possible, e.g., several images per second. The images are preferably captured consecutively at a repetition rate of at least 10 Hz, particularly preferably at least 20 Hz.

[0087] The illumination source 20 is used as a projector to project a variable pattern onto the object arranged in the measurement area. First, the first pattern 200 is projected from the illumination source 20 onto the face of the user 100. Specifically, the first pattern 200 is superimposed with the predetermined points of the object 105. The image data of the first image 300 is checked to determine whether one or all of the predetermined points are marked by a feature of the first pattern 200. As shown in Figure 6AHowever, as shown, none of the predetermined points are marked by feature M1 of the first pattern 200. This means that in the image data of the first image 300, none of the features of the first pattern 200 coincide with any of the predetermined points. In particular, the straight section of the contrast edge of feature M1 does not pass through any of the predetermined points, but is spaced apart from it. Therefore, the 3D coordinates of the predetermined points are not yet calculated from the image data of the first image 300.

[0088] Figure 5Bshows a second pattern 210, which is also emitted by the illumination source 20. After it is determined, as described above, that in the first image 300 the predetermined points are not all marked by a feature of the first pattern 200, an iteration of the method with a varied pattern is carried out. Here, the second pattern 210 can be generated from a variation of the first pattern 200. In the variation, for example, a partial area unlit in the first pattern 200 (in Fig. 5A shown on the left) is divided into two substantially rectangular new sub-regions of the second pattern 210, one of which is illuminated and the other of which is unilluminated.

[0089] The larger one (in Fig. 5AThe illuminated partial area of ​​the first pattern 200 (shown on the right) has been divided into four in the second pattern 210, namely into four essentially rectangular new partial areas of the second pattern 210. Of these four essentially rectangular new partial areas, two are illuminated and two are unilluminated in the second pattern 210. Overall, the second pattern 210 has three illuminated and three unilluminated, essentially rectangular areas. Adjacent partial areas and / or areas are always alternately illuminated and unilluminated. Therefore, contrast edges arise between said illuminated and unilluminated partial areas / areas, which are suitable as features of the pattern for marking predetermined points on the object 105. All of these contrast edges have, at least in sections, an essentially rectilinear area with which the predetermined points can be marked.

[0090] Overall, the patterns projected by the illumination source 20 can be essentially checkerboard-like. This means that the patterns exclusively comprise essentially rectangular subregions and / or partial areas that are either fully illuminated or unilluminated. Furthermore, the partial areas are alternately illuminated and unilluminated in a vertical and / or horizontal direction. This results in essentially horizontally arranged and essentially vertically arranged contrast edges as features of the variable pattern. This checkerboard-like arrangement can apply to all patterns projected into the measurement area by the illumination source 20. The individual "checkerboard fields," i.e., the illuminated and unilluminated subregions or partial areas of the pattern, can have different dimensions, e.g., as rectangles.

[0091] Such a projected light pattern can be constructed, for example, by drawing essentially perpendicular and essentially vertical lines over the two-dimensional emission surface or pattern surface, which essentially form rectangular fields.

[0092] Figure 6B shows a schematic representation of a second image 310 of the left eye 114, framed by the spectacle frame 101, onto which the second pattern 210 is projected. The second image 310 is converted into image data in which the predetermined points are again detected. Furthermore, and again, a check is performed to determine whether all predetermined points are marked by a respective feature of the second pattern 210. Since this is not yet the case in the second image 310, the projection pattern is varied again in a next step.

[0093] Figure 5Cshows the further variation using a third pattern 220, in which the number of illuminated and unilluminated sub-regions and / or sub-areas of the pattern arranged in a checkerboard pattern has been further increased. In this variation, both the number of geometric shapes contained in the pattern has been increased and their average size has been reduced. In this variation, the number of contrasting edges of the pattern has been increased. In this variation, both the number of essentially vertically aligned dividing lines through the pattern has been increased, as well as the number of essentially horizontally extending dividing lines between the individual sub-regions / sub-areas of the pattern have been increased.

[0094] Figure 6C shows a schematic representation of a third image 320 of the left eye 114 of the user 100, framed by the spectacle frame 101, onto which the Figure 5CThe third pattern 220 shown is projected. This image is also checked to see whether the predetermined points are each marked by a feature of the third pattern 220. If the check is negative, the pattern is further varied.

[0095] Figure 5D shows a schematic representation of a fourth pattern 230 based on a variation of the third pattern 220. The fourth pattern 230 does not necessarily arise directly from the third pattern 220. Between the generation of the third pattern 220 and the generation of the fourth pattern 230, further patterns not shown in the figures can be generated to capture additional images and projected into the measurement area. With each variation, the number of contrast edges of the pattern can be increased, particularly in the areas where the predetermined points of the object 105 are located.

[0096] Figure 6Dshows a schematic representation of a fourth image 330 of the eye framed by the spectacle frame 101, onto which the fourth pattern 230 is projected. As in Figure 6D As shown, in particular, the four left frame points OLF, TLF, ULF and MLF are each marked by a feature of the fourth pattern 230. In this case, a contrast edge of the fourth pattern 230 is arranged substantially perpendicular to the direction of travel of the spectacle frame 101. An intersection point between the spectacle frame 101 and the respective feature of the fourth pattern 230 serves to mark the respective predetermined point, i.e., in this case, the respective left frame point in the fourth image 330. From this marking, the respective associated 3D coordinate of the predetermined point can be determined and / or calculated using the previously known relationship of the illumination source 20 to the image recording device 10, e.g., with the aid of the respective assignment functions fp .

[0097] When varying the pattern, the calibration of the setup, i.e., the previously known relationship between the image recording device 10 and the illumination source 20, can be taken into account and / or utilized. Since the distance of the device to the object 105 is initially unknown, the distance is determined iteratively by pattern projection and locating the features of the respective projected pattern in the respectively recorded image. If the predetermined points are each superimposed with a feature of the projected pattern, and if the individual features of the projected pattern in the recorded image can be assigned to elements of the illumination source 20, e.g., their respective pixel(s), a sufficiently accurate determination of the 3D coordinates of the individual, predetermined points of the object 105 can be performed. All 3D coordinates can be calculated based on a single image, namely the most recently recorded image, in the exemplary embodiment, based on the fourth image 330.There is no need to solve the correspondence problem, i.e. finding corresponding points in two images taken from different angles.

[0098] In other words, the device is designed and provided to perform active and adaptive video centering, wherein a projected pattern is varied and, so to speak, tracked to the predetermined points of the object 105. By tracking the projection, an increase in spatial resolution is achieved in successive recordings, i.e., captured images. Stable methods of structured illumination based on pattern sequences can be used for this purpose.

[0099] To detect the predetermined points in the image and to check whether the respective predetermined points are marked by a feature of the projected pattern, known methods in image processing can be used. For face recognition and / or localization of facial features, i.e., to detect the predetermined points, cascade classifiers can be used (as described in more detail in the article "Fast Multi-view Face Detection" by M. Jones and P. Viola) or descriptor vectors, in which key points described by a descriptor are first determined, in order to then determine a match by comparison with a reference descriptor. Examples of descriptors include FAST, FREAK, BRISK, etc. These methods are generally known to those skilled in the art.

[0100] A movement of the user 100, i.e., a movement of the object 105, in successively recorded images can also be determined using key points and taken into account in the method. Alternatively or additionally, an optical flow calculation can also be performed to track the movement and take it into account for subsequent pattern projections.

[0101] By taking into account the movement of the object and completely determining all 3D coordinates from a single image, artifacts can be excluded that would otherwise occur in a sequence of consecutive, structured-illuminated images and could distort the result.

[0102] A variety of patterns can be used as the projection pattern and / or light pattern emitted by the illumination source 20, which can be used as structured illumination of the measurement area. The method does not aim at a dense and / or complete 3D reconstruction of the measurement object or its surface, but merely at the determination of individual predetermined points, e.g. up to 100 such selection points, preferably up to 50 such selection points, particularly preferably from 8 to 12 selection points. Therefore, a large-area, largely unstructured pattern can be projected as a first pattern in a first step. If or as soon as the at least one predetermined point is arranged within the projected pattern, a sequence for increasing the spatial resolution of the pattern can be started, i.e., with the variation of the projected pattern.This is typically the case when at least one predetermined point is positioned within the measurement area. The pattern(s) can be configured to substantially overlap the entire measurement area.

[0103] Possible variation sequences include binary codes and / or grey codes. Such codes are explained in more detail in the article "Structured-light 3D surface imaging: a tutorial" by J. Geng.

[0104] Although a static use of colors for coding may seem unfavorable at first glance, given that the variety of eyeglass frames brings with it an equally wide variety of colors, the projected pattern can also be color-coded. For example, a pattern color complementary to the background could be selected, especially for marking the eyeglass frame. This further increases the contrast in the captured image, which can improve the determination of the 3D coordinates of the predetermined points.

[0105] To mark elements with a small physical dimension, for example, to mark the frame rim of metal eyeglasses, a one-dimensional sequence of patterns can be used, such as a striped pattern. Here, the predetermined point on the frame rim is specifically marked by a contrasting edge of the striped pattern. In general, contrasting edges of the projected pattern should have a comparatively large intersection angle with the largest dimension of the area surrounding the predetermined point. For example, a substantially straight contrasting edge in the image can be substantially perpendicular to the direction of the respective section of the frame rim.In this case, detection and / or localization of the predetermined point can be enabled by extending the object element on which the predetermined point is located in a direction other than the direction in which the contrast edge of the pattern is located. In this case, a discontinuity of the contrast edge is recorded in the recorded image, such as in the feature sections M1', M1", and M1‴ of feature M1 in . Figure 4 shown.

[0106] The predetermined points are points of an object element and / or object feature, such as points on or at the edge of the spectacle frame and / or lens. Likewise, points such as the pupil center can be predetermined as a selection point for the corresponding pupil as an object element and / or object feature.

[0107] As described above, due to a lack of back-reflection of the projected pattern, the 3D coordinates of several surrounding points can be used to determine the 3D coordinates of the pupil centers.

[0108] The 3D coordinates of the predetermined points can be determined in parallel, i.e., cascading toward a single image, or sequentially. Predefined points that are spatially close to each other but could not be clearly identified, for example, due to discontinuities in the pattern, can be detected sequentially to avoid ambiguities. These successively determined 3D points can be assigned to other 3D points via 3D points determined in multiple images.

[0109] The projection of the pattern can be improved, for example, by taking into account the reflectivity at the predetermined points and / or global exposure effects.

[0110] The method does not require the computation of consecutively acquired images. Such computation can be omitted entirely. While consecutively acquired images can be compared to calculate object movement, particularly to calculate optical flow in the acquired images, a single, standard digital camera (as opposed to a 3D camera) can be used as the image capture device. In particular, only a single, two-dimensional image can and / or must be evaluated to calculate the 3D coordinates, in contrast to methods in which multiple images (e.g., with phase-shifted patterns) are compared with one another or 3D cameras are used.

[0111] The video centering described above can also be used, for example, in the dental or orthopedic field, where model-based surfaces or bodies are used that have a defined number of parameters to describe them.

[0112] Figure 7 A flowchart shows steps of a method for determining 3D coordinates of predetermined points of an object. The flowchart specifically shows steps that can be performed by a computer program product.

[0113] Initially, in step 400, the object is placed in the measurement area, which can be configured, for example, as a well-defined measurement volume. A starting pattern is selected as the first pattern (see also step 404), which is projected onto the object. In step 401, an image of the object onto which the starting pattern is projected is captured.

[0114] The captured image, or more precisely the digital image data of the captured image, is then analyzed in step 402. In this step, the image data is analyzed to detect the predetermined points in the captured image. The predetermined points are defined in the Figure 7 In the flowchart shown, these are referred to as selection points. Generally, these selection points are individual, selected, and predetermined points on the object whose 3D coordinates are to be determined.

[0115] Following image analysis 402, in a step 403, the individual selection points in the image data are checked to determine whether they are each marked with a feature of the pattern. If this is not yet the case, the projected pattern is varied in step 404, and the varied pattern is projected onto the object in the measurement area. This is followed in step 401 by capturing another image of the object onto which the varied pattern is projected. The process is iterated until it is determined in step 403 that all selection points are marked with a feature of the pattern. The measurement can then be concluded in step 405, and the 3D coordinates of all selection points, i.e., all points to be determined, can be determined from the last image captured or the associated image data. This is done based on the previously known relationship between the illumination device and the illumination source, e.g., based on the known assignment functions. fp .

[0116] The flowchart further shows two optional steps of the method, which can also be carried out within the framework of the method. In a first option, which is identified in the figure with the reference number 406, the temporal progression of the position of some or all of the selection points, i.e. the temporal progression of the predetermined points, can be tracked. This enables a prediction of where the selection points are likely to be located when the next image is captured. This can be taken into account when varying the pattern. Thus, a movement of the object in subsequent images can be predicted in order to adapt the projected pattern accordingly. This first option 406 is particularly useful in applications where an object is on a conveyor belt. In addition, slow oscillations, at least compared to the image capture rate, can also be compensated, e.g.Fluctuations of a person in front of a measuring device whose face is to be recorded. These fluctuations can be either compensatory movements or breathing movements.

[0117] As a further option of the method, a step 407 can be provided in which the selection points in the image are determined manually, for example by an operator. Such an operator can be an optician, for example. In a fully automatic determination of all selection points, a manual or semi-automatic selection of the selection points can therefore also take place. This can, for example, be a one-time process in which the user, for example an operator, only marks the predetermined points in the first image. The search for the predetermined points can then be carried out more easily and robustly in the subsequent images with computer or software support, in particular by means of robust, previously known image processing methods, such as pattern matching or correlating the image points with one another.

[0118] As in Figure 5A shown, can be used for the Figure 7For example, in the method shown, the first pattern 200 can be used as a static starting pattern. The first pattern 200 has only a single transition from light to dark or vice versa as feature M1, which can be clearly detected in the image data and assigned to a distance from the image recording device 10. A distance determination can be made from this single transition, for example, to estimate the distance of the test subject, i.e., the user 100. The distance determination can be carried out analogously to a light section method with a projected light line. The distance information in the vertical direction, which is obtained by the vertically arranged transition (feature M1) between the light and dark partial area of ​​the first pattern 200, can be transferred in a model-based manner from the measurement location to the location in the two-dimensional image at which a selection point was found by the image processing in step 402.

[0119] To transfer the distance information from the measuring location to the selection point, a model can be used as a first approximation, for example, which is designed as a plane arranged parallel to the camera plane.

[0120] In general, an average distance of the object 105 in the measurement range can be determined, for example, by determining an average value of the determined distances of the vertical line (feature M1) of the first pattern 200 in the recorded first image 300. This results in the average distance of the object 105 in the measurement range.

[0121] Alternatively, a 3D facial model for the object 105 can already be assumed and / or used and / or taken into account for a first distance determination. Thus, a selection point detected in the two-dimensional image, e.g., a point in the first image 300, can be assigned an analogous or associated point in the three-dimensional facial model. A difference or distance (e.g., in the projection direction of the image recording device 10, i.e., in the direction of the optical axis 11) from this detected selection point to the associated point in the three-dimensional facial model can be used as a distance correction value to better align the positioning of the features of the pattern with the selection point. When varying the pattern, a distance correction value based on a deviation of the detected selection point from an associated point in the facial model can thus be taken into account.In one embodiment, the distance correction value can also be determined from a 2D distance of the detected selection point from the associated point in the face model, i.e. from the image data of the recorded two-dimensional image.

[0122] From the distance determined in this way, or model distance in 3D, together with the selection points determined in two dimensions (see step 402), a new pattern can be calculated, for example the one in Figure 5B shown second pattern 210, and subsequently the one shown in Figure 5C The third pattern 220 shown here. Here, a pattern is selected in which edges of the pattern are placed onto the determined selection points by projection. The pattern is thus varied in such a way that features of the pattern are expected to pass through the locations of the selection points, which have so far only been determined in two dimensions.

[0123] In this case, it can be taken into account that the individual contrast edges of the pattern do not overlap in such a way that a clear assignment of the projected patterns to the individual features of the pattern is no longer possible. When varying the pattern, it can therefore be taken into account that the features of the projected pattern can be clearly assigned. This can be done by taking into account an expected change in distance from the image recording device along a lateral position of the object in the image (e.g. gradient from nose to eye socket) and / or by successively increasing the number of features (contrast edges) of the pattern. For example, when varying the pattern, only a single additional vertical and / or horizontal contrast edge can be arranged in a previously homogeneous sub-area of ​​the pattern per iteration step.

[0124] According to one embodiment, when the pattern is varied, a maximum of one new vertical and / or horizontal contrast edge is added per homogeneous surface area and / or sub-area. This reduces the risk of overlapping pattern features and thus the risk of reducing the unambiguous assignability in the recorded image.

[0125] As well as all selection points to be determined are covered by a feature, in particular an edge, of the pattern in the projection of the camera, as for example in Figure 6D As shown, the measurement can be completed. All 3D coordinates of the selection points can then be calculated using a single 2D image (see step 405 and fourth image 330).

[0126] According to one embodiment, the method serves to determine the 3D coordinates of one to fifty predetermined points (selection points), preferably of five to twenty selection points, particularly preferably of 10 to 16 selection points. List of reference symbols

[0127] 10Image pickup device 11Optical axis 20Illumination source 21Illumination direction 30Intersection point 100User 101Frame 102Lens rim / Frame rim 105Object 110Lens 112Right eye 114Left eye 120Boundary of the right eye in the box dimension 122Boundary of the left eye in the box dimension 200 first pattern 210 second pattern 220 third pattern 230 fourth pattern 300 first image 310 second image 320 third image 330 fourth image 400-407Procedure step A Pattern B Pattern C Pattern LPM left pupil center RPM right pupil center NRF nasal right socket point TRF temporal right socket point NLF nasal left socket point TLF temporal left socket point ORF upper right socket point URF lower right socket point OLF upper left socket point ULF lower left socket point HR horizontal plane through the right pupil center VR vertical plane through the right pupil center HL horizontal plane through the left pupil center VL vertical plane through the left pupil center R1, R2, R3 features in the right eye L1, L2, L3, L4 features in the left eye M1 feature M1', M1", M1‴ feature section

Claims

1. Method for determining 3D coordinates of at least one predetermined point (LPM; RPM; ORF; NRF; URF; TRF; OLF; TLF; ULF; NLF) of an object (105), wherein the object (105) is a head of a user (100) with a spectacle frame (101) arranged thereon in a position of use, and wherein a point of the object (105) which is suitable for calculating optical parameters is used as the at least one predetermined point (LPM; RPM; ORF; NRF; URF; TRF; OLF; TLF; ULF; NLF) is used as the at least one predetermined point (LPM; RPM; ORF; NRF; URF; TRF; OLF; TLF; ULF; NLF) of the object (105) that is usable for calculating optical parameters, and wherein: a) the object (105) is arranged in a measuring range, b) a variable light source (20) projects a variable pattern (A; B; C; 200; 210; 220; 230) onto the object (105) arranged in the measuring range, c) an image recording device (10), which is arranged in a known relationship with the illumination source (20), records an image (300; 310; 320; 330) of at least a partial area of the object (105) illuminated by the variable illumination source (20), d) in the recorded image (300; 310; 320; 330), at least one predetermined point (LPM; RPM; ORF; NRF; URF; TRF; OLF; TLF; ULF; NLF) is detected, e) the 3D coordinates of the at least one predetermined point (LPM; RPM; ORF; NRF; URF; TRF; OLF; TLF; ULF; NLF) from the recorded image (300; 310; 320; 330) taking into account the known relationship between the image recording device (10) and the light source (20), if a check of the recorded image (300; 310; 320; 330) shows that the at least one predetermined point (LPM; RPM; ORF; NRF; URF; TRF; OLF; TLF; ULF; NLF) in the recorded image (300; 310; 320; 330) is marked by a feature (M1; L1; L2; L3; R1; R2; R3) of the variable pattern (A; B; C; 200; 210; 220; 230), and f) if the check of the recorded image (300; 310; 320; 330) reveals that the at least one predetermined point (LPM; RPM; ORF; NRF; URF; TRF; OLF; TLF; ULF; NLF) in the recorded image (300; 310; 320; 330) is not marked by a feature (M1; L1; L2; L3; R1; R2; R3) of the variable pattern (A; B; C; 200; 210; 220; 230) that is not marked by a feature (M1; L1; L2; L3; R1; R2; R3) of the variable pattern (A; B; C; 200; 210; 220; 230) projected by the illumination source (20) onto the object (105) is varied and then steps c) to e) are repeated under illumination with the varied pattern (A; B; C; 200; 210; 220; 230).

2. Method according to claim 1, wherein the variable pattern (A; B; C; 200; 210; 220; 230) projected by the illumination source (20) onto the object (105) is varied and then steps c) to e) are repeated with the varied pattern (A; B; C; 200; 210; 220; 230) until the at least one predetermined point (LPM; RPM; ORF; NRF; URF; TRF; OLF; TLF; ULF; NLF) in the recorded image (300; 310; 320; 330) is marked by a feature (M1; L1; L2; L3; R1; R2; R3) of the variable pattern (A; B; C; 200; 210; 220; 230) and the 3D coordinates of the at least one predetermined point (LPM; RPM; ORF; NRF; URF; TRF; OLF; TLF; ULF; NLF) are calculated.

3. Method according to claim 1 or 2, wherein the variable pattern (A; B; C; 200; 210; 220; 230) is specifically varied such that the varied pattern (A; B; C; 200; 210; 220; 230) is expected to correspond to the at least one predetermined point (LPM; RPM; ORF; NRF; URF; TRF; OLF; TLF; ULF; NLF) of the object (105) with a feature (M1; L1; L2; L3; R1; R2; R3) of the varied pattern (A; B; C; 200; 210; 220; 230).

4. Method according to one of the preceding claims, wherein successive images (300; 310; 320; 330) are recorded at a repetition frequency of at least 10 Hz until the 3D coordinates of the at least one predetermined point (LPM; RPM; ORF; NRF; URF; TRF; OLF; TLF; ULF; NLF) are calculated.

5. Method according to one of the preceding claims, wherein the variable pattern (A; B; C; 200; 210; 220; 230) has at least one substantially horizontal and / or vertical contrast edge as a feature (M1; L1; L2; L3; R1; R2; R3) which is displaced when the pattern (A; B; C; 200; 210; 220; 230) varies.

6. Method according to one of the preceding claims, wherein the variable pattern (A; B; C; 200; 210; 220; 230) has a number of geometric shapes with contrast edges as features (M1; L1; L2; L3; R1; R2; R3) and, when the pattern (A; B; C; 200; 210; 220; 230) is varied, the number of geometric shapes and thus the number of contrasting edges of the pattern (A; B; C; 200; 210; 220; 230) is increased.

7. Method according to one of the preceding claims 8, wherein at least one of the following points is used as the at least one predetermined point (LPM; RPM; ORF; NRF; URF; TRF; OLF; TLF; ULF; NLF): - a pupil center (LPM; RPM), - an inner temporal frame point (TRF; TLF), - an inner nasal frame reference (NLF), - an inner reference point above the pupil (ORF; OLF) and / or - an inner frame reference point below the pupil (URF; ULF).

8. Method according to one of the preceding claims, wherein optical parameters of the user (100) are determined from the determined 3D coordinates of the at least one predetermined point (LPM; RPM; ORF; NRF; URF; TRF; OLF; TLF; ULF; NLF).

9. Method according to one of the preceding claims, wherein the 3D coordinates of a pupil center (LPM; RPM) of the user (100) are determined by marking at least two points adjacent to the pupil center (LPM; RPM) as predetermined points, each with at least one feature (L1; L2; L3; R1; R2; R3) of the variable pattern (A; B; C; 200; 210; 220; 230) are marked as predetermined points, from whose 3D coordinates the 3D coordinates of the pupil center (LPM; RPM) are estimated.

10. Device for determining 3D coordinates of at least one predetermined point (LPM; RPM; ORF; NRF; URF; TRF; OLF; TLF; ULF; NLF) of an object (105), wherein the object (105) is a head of a user (100) with a spectacle frame (101) arranged thereon in a position of use, and the at least one predetermined point (LPM; RPM; ORF; NRF; URF; TRF; OLF; TLF; ULF; NLF) is a point of the object ( ) that can be used to calculate optical parameters, comprising: - a variable light source (20) for projecting a variable pattern (A; B; C; 200; 210; 220; 230) onto the object (105) arranged in a measuring range, - an image recording device (10) arranged in a known relationship with the illumination source (20) for recording an image (300; 310; 320; 330) of at least a partial area of the object (105) illuminated by the variable illumination source (20), - a detection module for detecting the at least one predetermined point (LPM; RPM; ORF; NRF; URF; TRF; OLF; TLF; ULF; NLF) in the recorded image (300; 310; 320; 330), - a coordinate determination module for determining the 3D coordinates of the at least one predetermined point (LPM; RPM; ORF; NRF; URF; TRF; OLF; TLF; ULF; NLF) from the recorded image (300; 310; 320; 330) taking into account the known relationship between the image recording device (10) and the light source (20), if a check of the recorded image (300; 310; 320; 330) shows that the at least one predetermined point (LPM; RPM; ORF; NRF; URF; TRF; OLF; TLF; ULF; NLF) in the captured image (300; 310; 320; 330) is marked by a feature (M1; L1; L2; L3; R1; R2; R3) of the variable pattern (A; B; C; 200; 210; 220; 230), and - a pattern variation module which, if the check of the recorded image (300; 310; 320; 330) reveals that the at least one predetermined point (LPM; RPM; ORF; NRF; URF; TRF; OLF; TLF; ULF; NLF) in the recorded image (300; 310; 320; 330) is not marked by a feature (M1; L1; L2; L3; R1; R2; R3) of the variable pattern (A; B; C; 200; 210; 220; 230) that varies the variable pattern (A; B; C; 200; 210; 220; 230) projected by the illumination source (20) onto the object (105).

11. Device according to claim 10, with a parameter determination device which is designed to determine optical parameters of a user (100) from the determined 3D coordinates.

12. Computer program product comprising program parts which, when loaded into a computer, are designed to carry out and / or control a method according to any one of claims 1 to 9.