Apparatus and method for capturing images of eyeglasses - Patent Application 20070122997

Abstract

The present invention relates to a device (1) for acquiring images of eyeglasses, comprising a flat surface defined by a contour (2) of known shape and dimensions, characterized in that the flat surface comprises at its periphery at least: a corrective means 3, a size standard 4, and in the central part: an acquisition area (6) having a homogeneous non-reflective color, comprising at least two openings (7, 7') for supporting the temples of said eyeglasses, and at least one position marker (8) representing the pupil of the wearer, facilitating the vertical positioning of the eyeglass lenses in said acquisition area (6).

Description

[Technical Field]

[0001] The present invention relates to the field of augmented reality simulation for optics, and more particularly to the field of virtual fitting of eyeglasses. [Background technology]

[0002] In the field of virtual fitting, it is known to use augmented reality to allow a customer to remotely try on a number of pairs of glasses that have been pre-modeled in 3D.

[0003] However, to enable this type of fitting, a 3D modeling work is required in advance for each pair of glasses. Various modeling techniques exist, such as manual work, CAD model modification, image acquisition, and 3D scanning, but using these techniques generally requires the use of many resources (tools, time, etc.). For example, in the case of 3D modeling by image acquisition, on the one hand, it is necessary to acquire images of the glasses under specific conditions (brightness, position of the glasses, etc.) and at several angles (images from the front of the glasses, images of the temples), and on the other hand, processing these images using complex modeling tools requires the intervention of experts in the field.

[0004] Regarding the image capture stage, several portable and low-cost devices already exist that allow for image capture of objects (e.g., portable photo booths). This type of device is known to be easy to use, but is not suitable for eyeglass-type products (e.g., precise positioning is not possible).

[0005] Regarding the image processing stage, the need for complex modelling tools and the need for the intervention of experts in the field results in high costs for the optician / optician.

[0006] Furthermore, 3D modeling of eyeglasses is a time-consuming process, so there is a significant time lag between when a new model of eyeglasses arrives in stores and when a virtual fitting becomes available.

[0007] There are also known devices or methods such as those described in Patent Documents 1, 2 and 3. However, these devices or methods are not satisfactory. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] US Patent Application Publication No. 2014 / 043332 [Patent Document 2] US Patent Application Publication No. 2016 / 327814 [Patent Document 3] US Patent Application Publication No. 2015 / 055085 Summary of the Invention [Problem to be solved by the invention]

[0009] The object of the present invention is therefore to propose an apparatus and method for acquiring images of spectacles that overcomes the above-mentioned drawbacks, is very low-cost, simple and fast to use, and allows the acquisition of images that can be used for augmented reality fitting by people who are not experts in the field. [Means for solving the problem]

[0010] Thus, according to the present invention, there is provided an apparatus for acquiring an image of spectacles comprising a flat surface defined by a contour having known shape and dimensions, said flat surface being characterised in that it comprises at least the following on its periphery: - corrective means, -Size standards, At the center: an acquisition area having a homogenous non-reflective color and including at least two openings for supporting said pair of temples and at least one position marker representing the wearer's pupils, facilitating vertical positioning of the spectacle lenses in said acquisition area.

[0011] The term "corrective means" refers to means used to obtain a fully frontal image of an object and can be of two types: those used upstream of image capture: for example, positioning supports for the image acquisition device; and those used downstream of image capture: for example, markers used to process the image and remove perspective effects.

[0012] A "size standard" is a measure of known size that allows one to find the size scale of an object in an image.

[0013] The term "wearer" refers to an individual who wears the eyeglasses on their face.

[0014] Preferably, the flat surface has a rectangular outline and has a colorimetric pattern.

[0015] Advantageously, the correction means and size reference is a black frame showing said outline of known dimensions.

[0016] Preferably, the colorimetric target is located between the black frame and the acquisition area.

[0017] Preferably, the opening is provided with a color adjustment means that is the same color as the acquisition area, allowing said opening to fit various types of eyeglass temples.

[0018] "Adjustment means" refers to all means used to restrict the periphery of the opening to a single space sufficient for the temples of the spectacles to pass through.

[0019] More preferably, the adjustment means comprises at least two U-shaped covers, the width of the opening of which may vary depending on the width of the eyeglass arms, and a rectangular cover, intended to be placed in front of the opening to limit its surface.

[0020] Advantageously, the acquisition area contains coded identification means that enable information about said device to be acquired.

[0021] The present invention also relates to a method for acquiring an image of spectacles using the device described above, characterized in that it comprises at least the following steps: a) Identification phase on the expert terminal: a1) displaying the retrieved web page on the output peripheral of the expert terminal; a2) Identification of experts; b) Acquisition stage: b1) placing the eyeglasses on the acquisition area of ​​the device (1); b2) capturing an image of the flat surface of the device (1) supporting the glasses using an image acquisition peripheral of the expert terminal; b3) Obtaining coded identification means; c) Image processing stage: c) Extraction of eyeglass frame (rack); c2) sending a cropped image of the eyeglasses and characteristics of the eyeglasses to a virtual fitting server; c3) Server-generated 3D model of glasses from the cropped image; c4) Generation of a virtual fitting web page by the expert terminal; d) Fitting stage: d1) sending a virtual fitting web page to the client's terminal; d2) Capturing an image of the customer's face using an image capture peripheral on the customer's device or loading an image of the customer's face from a photo gallery; d3) Fitting the 3D model of the eyeglasses onto the image of the customer's face to form a fitting image.

[0022] A "coded identification means" is a code or visual means that is unique to the eyeglasses (e.g., a QR code).

[0023] A "cut-out image of glasses" is an image of glasses in which the background of the image has been completely removed, leaving only the glasses.

[0024] Preferably, after step a2, there is a step a3: searching for a 3D model of the glasses in a catalogue present on a server: If the glasses are not present in the catalogue: after step c3, there is a step c3': saving the 3D model of the glasses in the catalogue on the server, - If the glasses are present in the catalogue: the image acquisition and processing stage is not performed and the fitting stage is performed from the 3D model of the glasses in the catalogue.

[0025] Even more preferably, step c1 comprises the following sub-steps: c11) correction of the image of the flat surface of the device (1) supporting the glasses; c111) detection of corrective measures; c112) Visual field correction, c113) Extracting the image area containing glasses, c12) Cutouts for eyeglass frames, c121) Remove colored background, c122) Resolution normalization.

[0026] Other advantages and features are described below. [Brief explanation of the drawings]

[0027] [Figure 1] FIG. 1 is a front view of the device according to the present invention; [Figure 2] Schematic diagram of the device and image acquisition device according to the invention [Figure 3] Flowchart of the method according to the invention [Figure 4] Schematic diagram of a two-cover system type adjustment means with two degrees of freedom for the opening of the device according to the invention. [Figure 5] Schematic diagram of a four-cover system type adjustment means with one degree of freedom for the opening of the device according to the invention [Figure 6] Schematic diagram of a three-cover system type adjustment means for the opening of the device according to the invention DETAILED DESCRIPTION OF THE INVENTION

[0028] Thus, according to figures 1 to 6, there is provided a device 1 for acquiring images of spectacles comprising a flat surface defined by a contour 2 of known shape and dimensions, said flat surface being characterised in that it comprises at least the following on its periphery: - corrective means 3, -Size standard 4, and At the center: an acquisition area 6 having a homogeneous, non-reflective color and including at least two openings 7, 7' for supporting said pair of temples and at least one position marker 8 representing the wearer's pupil, facilitating the vertical positioning of the spectacle lens in said acquisition area 6.

[0029] Regardless of the wearer, their pupils are at approximately the same height as the spectacle lenses when the glasses are worn. This height can be embodied by a line called the "line of sight."

[0030] The color of the acquisition area 6 is selected according to the color of the glasses, with the aim of making the color of the glasses stand out as much as possible from the background color of the acquisition area 6. Advantageously, the color of the acquisition area 6 is an anti-reflective (matt) background color. These choices of color for the acquisition area 6 make it easier to cut out. Preferably, the flat surface has a rectangular contour 2 and carries a colorimetric pattern 5.

[0031] The flat surface is preferably A4 size for easy transport and may have feet 10-15cm high to allow sufficient space for the temples of the glasses to be held in a horizontal position.

[0032] Advantageously, the correction means 3 and size reference 4 are a black frame showing the contour 2 of known dimensions. The fact that the black frame has known dimensions (length and width) makes it easy to correct an image that was not captured correctly from the front. Indeed, if the front image is not captured correctly, the dimensions of the frame measured on the image will no longer be the same. Therefore, the image can be warped to obtain the frame with its original dimensions. Thus, several parameters of the image (orientation, field of view, etc.) are corrected, and a complete front view of the device 1 can be obtained. In some cases, other means for correction can be used, such as installing the image acquisition device on a specific support, which allows adjusting its positioning relative to the flat surface and controlling the angle of view. Furthermore, since its actual dimensions are known, this black frame also serves as a size 4 or scale reference. Other size 4 references (e.g., graduated ruler, checkerboard, etc.) can be used. Finally, this frame perfectly demarcates the acquisition support 6 by marking the contour 2 on its black flat surface, facilitating image cropping.

[0033] Preferably, a colorimetric pattern 5 is placed between the black frame and the acquisition area 6. This is captured by the image acquisition device and serves as a reference for correcting color deviations observed between the color of the real object and the color of the object in the image. These observed color deviations may be related in particular to the lighting during the photographic capture.

[0034] Preferably, the openings 7, 7' include adjustment means 9, 9a, 9b, 9c, 9d, 9e, 9f, 9g, 9h, 9i of the same color as the acquisition area 6, allowing the openings 7, 7' to accommodate various types of eyeglass temples. Various eyeglasses on the market have temples of various shapes and thicknesses, which can be attached to the top of the frame at 1 / 3-2 / 3 of the frame height or in the center. To support these various temples, the openings 7, 7' are designed to have a fairly large perimeter of approximately 30 mm x 30 mm for rectangular openings 7, 7'. The large size of these openings 7, 7' can cause difficulties during the cutting process, and for this reason, adjustment means 9, 9a, 9b, 9c, 9d, 9e, 9f, 9g, 9h, 9i of the same color as the acquisition area 6 are used.

[0035] These adjustment means 9, 9a, 9b, 9c, 9d, 9e, 9f, 9g, 9h, 9i limit the openings 7, 7' only to a space sufficient for the temples of the eyeglasses to pass through. Even more preferentially, the adjustment means 9, 9a, 9b, 9c, 9d, 9e, 9f, 9g, 9h, 9i include at least two U-shaped covers 9g, 9h and a rectangular cover 9i, the width of whose openings may vary depending on the width of the temples of the eyeglasses, and are intended to be positioned in front of the openings 7, 7' to limit their surface. The slots or openings of these covers 9g, 9h can have three different widths (4, 7, or 8 mm) to fit all commercially available frames. The U-shaped covers 9g, 9h limit the height and width of the openings 7, 7' depending on their shape, with two degrees of freedom. The rectangular cover 9i limits only the height, with one degree of freedom. It is possible to envisage other adjustment means 9, 9a, 9b, 9c, 9d, 9e, 9f, 9g, 9h, 9i having the same colour as the acquisition area 6, for example as follows: - Weft: Two weft threads are superimposed perpendicularly (i.e., frame oriented at 90° from one another) -Loose materials: sand or gel type materials that the temples sink into; -Fabrics with buttonholes or star-shaped slots, a cover system 9, 9a with two degrees of freedom: each opening 7, 7′ can be covered by two covers 9, 9a, the first L-shaped 9a and the second L-shaped 9b being rotated clockwise by 90°; - 4-cover system 9b, 9c, 9d, 9e, 9f with one degree of freedom: two covers are tooth type 9c, 9d and two are ratchet type 9e, 9f, various covers 9b, 9c, 9d, 9e, 9f of complementary shape are arranged in a mirrored manner and stacked 2x2, the remaining opening has a diamond shape.

[0036] Advantageously, the acquisition area 6 comprises coded identification means 10 making it possible to acquire information about said device 1. This identification means may be, for example, a QR code, which makes it possible, for example, to know the dimensions of the device 1.

[0037] The present invention also relates to a method 100 for acquiring an image of spectacles using the device 1 described above, characterized in that it comprises at least the following steps: a) Identification phase P1 on the expert terminal A1: a1) displaying 110 the retrieved web page on the output peripheral of the expert terminal A1; a2) Expert Identification 120; b) Acquisition Phase P2: b1) placement 130 of the glasses on the acquisition area 6 of the device 1; b2) capturing 140 an image of the flat surface of the device 1 supporting the spectacles using the image acquisition peripheral of the expert terminal A1; b3) obtaining 150 the coded identification means 10; c) Image processing stage P3: c) Extraction of eyeglass frames 160; c2) sending a cropped image of the eyeglasses and characteristics of the eyeglasses to the virtual fitting server 170; c3) Server-generated 3D model of glasses from cropped image 180; c4) Generation of a virtual fitting web page 190 by the expert terminal A1; d) Fitting stage P4: d1) sending 200 a virtual fitting web page to the client terminal; d2) capturing an image of the customer's face using an image capture peripheral of the customer terminal or loading an image of the customer's face from a photo gallery 210; d3) Fitting 220 the 3D model of the eyeglasses onto the image of the customer's face to form a fitting image.

[0038] Preferably, there is a step c1' between c1, 160 and c2, 170: visual confirmation of the extraction quality. During this step, an expert checks the photography, the presence of diffuse reflections, the quality of the crop, etc. It is also possible to display the glasses superimposed on the face and manually change the frame settings (mainly the height) as well as manually correct the color tone (saturation, etc.).

[0039] Preferably, after step a2, 120, there is a step a3: searching for a 3D model of the glasses in a catalogue present on a server: If the glasses are not in the catalog: after step c3, 180, there is a step c3': saving the 3D model of the glasses in the catalog on the server, - If the glasses are present in the catalogue: the image acquisition step P1 and the processing step P2 are not performed and the fitting step P4 is performed from the 3D model of the glasses in the catalogue.

[0040] The expert terminal A1 and the customer terminal may be a computer, a tablet, or a mobile phone, and may be considered to communicate via the Internet.

[0041] Even more preferably, step c1, 160 comprises the following sub-steps: c11) correction of the image of the flat surface of the device (1) supporting the glasses 161; c111) Detection of corrective measures 161a; c112) Visual field correction 161b, c113) Extraction of image regions containing glasses 161c, c12) Cutout of eyeglass frame 162, c121) Removal of colored background 162a, c122) Resolution normalization 162b.

[0042] The overall correction process 161 involves the detection of a black border on the captured image, the deformation of the image to obtain a black border with the same dimensions as the black border of the support, the cropping of the black border to obtain an image of known dimensions (the image of the border dimensions), and the trimming of the region of interest (the metric size of the region of interest is fixed and depends on the mesh used for the 3D model). The processing operations used in this step are: morphological filtering, adaptive thresholding, contour detection, contour selection based on the constraints of the device 1, and homography. For cropping 162, the overall process is as follows: background extraction and transparent surface generation from the image obtained in the previous step, and image resizing to obtain the final texture within the resolution constraints. The algorithm used for extraction is based on distance measurement in the HSV color space. Depending on the distance, a threshold is applied while managing an anti-aliasing ramp at the boundaries. Other algorithms could potentially be used: thresholding, watershed, floodfill, histogram backprojection, grabcut, contour detection, etc.

[0043] Preferably, after step c12, 162, there is an automatic correction step c13 that allows for the correction of errors (such as non-horizontal captured frames) related to the capture step b2, 140. During this step, the cropped frame can be subjected to a slight rotation (so that the glasses are well positioned along a horizontal plane) or horizontal centering.

[0044] Advantageously, step c1, 160 is performed by an artificial intelligence algorithm. For certain types of eyeglasses, sunglasses, or eyeglasses with invisible frames, the cropping step 160 can prove to be very difficult. This step 160 can therefore be considered to be performed by an artificial intelligence algorithm trained on a catalog of several models of eyeglasses. The algorithm therefore becomes increasingly efficient. In this case, the cropping is performed by a server rather than by the expert terminal A1.

[0045] The method 100 may further provide a step of extracting the spectacle lens, the purpose of which is to delimit the area occupied by the spectacle lens and to apply deformations to this area in order to simulate the effect that the spectacle lens has. In fact, depending on the correction prescribed by the ophthalmologist, the spectacle lens will undergo deformations in the area where it is located. Thus, the rendering of the fitting will be more realistic. Finally, the invention relates to an image acquisition system, characterized in that it includes at least: an image acquisition device 1 as described above, a customer terminal in communication with the specialist terminal A1, including at least one input / output peripheral device that allows at least the implementation of the fitting phase P4 of the method 100 described above; an expert terminal A1 communicating with said client terminal and with the server and including at least one input / output peripheral device enabling at least the implementation of steps c1, c2 and c4, 160, 170, 180 of the identification phase P1, the acquisition phase P2 and the image processing phase P3 of the method 100 described above, a virtual fitting server in communication with the expert terminal A1 and designed to implement at least step c3, 180 of the image processing stage P3 of the method 100 described above.

[0046] The expert terminal A1 and the customer terminal may be a computer, a tablet or a mobile phone. It is assumed that communication between the various entities of the system is established via the Internet.

[0047] The image acquisition method 100 is used in conjunction with the image acquisition device 1 described above. However, the method can be used with any type of support known to those skilled in the art.

[0048] The device 1 and method 100 for acquiring images of eyeglasses according to the present invention find particular application in the virtual fitting of eyeglasses, however, it will be clear that the device 1 and method 100 for acquiring images of eyeglasses can be adapted and used for acquiring images of other types of instruments.

[0049] Finally, it should be understood that the above-described examples of the device 1 and method 100 for acquiring images of eyeglasses according to the present invention are specific examples only and do not constitute any limitation on the present invention. Other embodiments 1. An apparatus (1) for acquiring images of eyeglasses including a flat surface defined by a contour (2) of known shape and dimensions, said flat surface having at least the following around its periphery: - corrective means (3), - size standards (4), and At the center: an acquisition area (6) having a homogeneous, non-reflective color and including at least two openings (7, 7') for supporting the temples of the eyeglasses and at least one position marker (8) representing the pupil of the wearer, facilitating the vertical positioning of the eyeglass lenses in the acquisition area (6); An apparatus (1) comprising: 2. The device (1) described in embodiment 1, characterized in that the flat surface has a rectangular contour (2) and has a colorimetric pattern (5). 3. The device (1) according to any of the first to third embodiments, characterized in that the correction means (3) and the size standard (4) are black frames marking the contour (2) of known dimensions. 4. The device (1) according to embodiment 2, characterized in that the colorimetric pattern (5) is arranged between the black frame and the acquisition area (6). 5. The device (1) according to any one of the first to fourth embodiments, characterized in that the openings (7, 7') comprise adjustment means (9, 9a, 9b, 9c, 9d, 9e, 9f, 9g, 9h, 9i) of the same color as the color of the acquisition area (6), allowing the openings (7, 7') to be adapted to various types of eyeglass temples. 6. The device (1) according to embodiment 5, characterized in that the adjustment means (9, 9a, 9b, 9c, 9d, 9e, 9f, 9g, 9h, 9i) comprise at least two U-shaped covers (9g, 9h), the width of whose openings can vary depending on the width of the temples of the glasses, and a rectangular cover (9i), which are arranged in front of the openings (7, 7') to limit their surface. 7. Device (1) according to any of the first to sixth embodiments, characterized in that the acquisition area (6) comprises coded identification means (10) that enable information relating to the device (1) to be acquired. 8. A method (100) for acquiring an image of eyeglasses using the device (1) according to any one of the first to seventh embodiments, comprising: At least one of the following steps: a) Identification phase (P1) on the expert terminal (A1): a1) displaying (110) the obtained web page on an output peripheral of said expert terminal (A1); a2) identification of said expert (120); b) Acquisition Phase (P2): b1) placing (130) eyeglasses on the acquisition area (6) of the device (1); b2) capturing (140) an image of the flat surface of the device (1) supporting the spectacles using an image capture peripheral of the expert terminal (A1); b3) obtaining (150) a coded identification means (10); c) Image Processing Stage (P3): c1) Extraction of the frame of the glasses (160); c2) sending a cropped image of the eyeglasses and characteristics of the eyeglasses to an image acquisition server (170); c3) generation by the server of a 3D model of the glasses from the cropped image (180); c4) generation of a virtual fitting web page by the expert terminal (A1) (190); d) Fitting stage (P4): d1) sending the virtual fitting web page to the client terminal (200); d2) capturing an image of the customer's face using an image capture peripheral of the customer terminal or loading an image of the customer's face from a photo gallery (210); d3) Fitting the 3D model of the eyeglasses onto the image of the customer's face to form a fitting image (220). A method (100) comprising: 9. After step a2 (120), step a3: searching for a 3D model of the eyeglasses in a catalogue existing on the server exists; If the glasses are not present in the catalogue: after step c3 (180), there is a step c3': saving a 3D model of the glasses in the catalogue on the server, If the glasses are present in the catalogue: the image acquisition step (P1) and the processing step (P2) are not performed and the fitting step (P4) is performed from the 3D model of the glasses in the catalogue. 9. The method (100) of embodiment 8, 10. Step c1 (160) comprises the following substeps: c11) correction of the image of the flat surface of the device (1) supporting the glasses (161); c111) detection of corrective measures (161a); c112) Visual field correction (161b), c113) Extraction of image regions containing glasses (161c), c12) Cutouts of eyeglass frames (162), c121) Removal of colored background (162a), c122) Resolution normalization (162b) 10. The method of embodiment 8 or 9, comprising:

Claims

1. An apparatus (1) for acquiring images of eyeglasses comprising a flat surface defined by a contour (2) of known shape and dimensions, said flat surface having at least the following around its periphery: a black frame marking said contour (2) with known dimensions, and At the center, an acquisition area (6) having a homogeneous, non-reflective color and including at least two openings (7, 7') for supporting the temples of said spectacles and at least one position marker (8) representing the pupil of the wearer, facilitating the vertical positioning of the spectacle lenses in said acquisition area (6); An apparatus (1) comprising:

2. 2. Device (1) according to claim 1, characterized in that the flat surface has a rectangular contour (2) and carries a colorimetric pattern (5).

3. 3. Device (1) according to claim 2, characterized in that the colorimetric pattern (5) is arranged between the black frame and the acquisition area (6).

4. The device (1) according to any one of claims 1 to 3, characterized in that the openings (7, 7') comprise adjustment means (9, 9a, 9b, 9c, 9d, 9e, 9f, 9g, 9h, 9i) of the same color as the color of the acquisition area (6), allowing the openings (7, 7') to be adapted to various types of eyeglass temples.

5. 5. The device (1) according to claim 4, characterized in that the adjustment means (9, 9a, 9b, 9c, 9d, 9e, 9f, 9g, 9h, 9i) comprise at least two U-shaped covers (9g, 9h), the width of whose openings can vary depending on the width of the temples of the glasses, and a rectangular cover (9i), which are arranged in front of the openings (7, 7') to limit their surface.

6. A method (100) for acquiring images of spectacles using a device (1) according to any one of claims 1 to 5, comprising: At least one of the following steps: a) Identification phase (P1) on the expert terminal (A1): a1) displaying (110) the acquired web page on an output peripheral of said expert terminal (A1); a2) Identification of said expert (120); b) Acquisition Phase (P2): b1) placing (130) eyeglasses on the acquisition area (6) of the device (1); b2) capturing (140) an image of the flat surface of the device (1) supporting the spectacles using an image capture peripheral of the expert terminal (A1); b3) Obtaining (150) a coded identification means (10); c) Image Processing Stage (P3): c1) Extraction of the eyeglass frame (160); c2) sending a cropped image of the eyeglasses and characteristics of the eyeglasses to an image acquisition server (170); c3) generation by the server of a 3D model of the glasses from the cropped image (180); c4) generation (190) of a virtual fitting web page by said expert terminal (A1); d) Fitting Phase (P4): d1) sending said virtual fitting web page to the client terminal (200); d2) capturing an image of the customer's face using an image capture peripheral of the customer terminal or loading an image of the customer's face from a photo gallery (210); d3) Fitting the 3D model of the eyeglasses onto the image of the customer's face to form a fitting image (220). A method (100) comprising:

7. After step a2 (120), there is step a3: searching for a 3D model of the eyeglasses in a catalogue present on the server; - if the glasses are not present in the catalogue: after step c3 (180), there is a step c3': saving a 3D model of the glasses in the catalogue on the server, If the glasses are present in the catalogue: the acquisition step (P2) and the image processing step (P3) are not performed and the fitting step (P4) is performed from the 3D model of the glasses in the catalogue.

7. The method (100) of claim 6.

Images