Method for adjusting color of at least one active lens of a smart spectacle, and corresponding ophthalmic system

Abstract

The invention provides a method implemented by computer means for adjusting color of at least one active lens of a smart spectacle (1) when used by a wearer, said method comprising: acquiring (100) at least one image representing said wearer and its current environment; determining (110) a first set of N colors, N being an integer greater or equal to 1, said first set of N colors comprising dominant colors in said at least one acquired image; determining (120) a second set of M harmonized colors, M being an integer greater or equal to 1, based on said first set of N colors; selecting (130) at least one harmonized color within said second set of M harmonized color; and controlling (140) said at least one active lens so that an external color of said at least active lens corresponds to the selected at least one harmonized color.

Description

[0001] Method for adjusting color of at least one active lens of a smart spectacle, and corresponding ophthalmic system

[0002] FIELD OF THE DISCLOSURE

[0003] The present disclosure relates to a method for adjusting color of at least one active lens of a smart spectacle, and to a corresponding ophthalmic system.

[0004] BACKGROUND OF THE DISCLOSURE

[0005] Generally, the choice to purchase ophthalmic devices such as spectacles tends to be based on esthetic and fashion trend. In most of the cases, when a customer wants to buy spectacles, especially sunglasses, the tint of the lenses is generally already defined. Most sunglasses are chosen according to the frame aesthetic (color and design) rather than the solar filter applied. Consequently, most sold sun lenses are grey, brown and grey-green. Colored lenses are chosen in an optical shop, based on the wearer's preference but without any advice or input from his habits. Besides, this choice may turn out to be irrelevant according to the wearer's outfit or environment.

[0006] For spectacles equipped with passive lenses, the customer needs to have several spectacles if he wants to adapt his look to the environment.

[0007] More recently, smart spectacles with electrochromic (or active) lenses have been developed which enable an automatic change in the tint of the lenses by applying control signals generated by the frame to activate electrochromic dyes in the lenses. For instance, document WO 2018 153 878 disclose several embodiments to achieve variable color transmissive electrochromic lenses and document US 2018 239 170 discloses solutions for providing reflective EC lens using for instance liquid crystal in Cholesteric phase, having pitch determined according to the wavelength to be filtered (Bragg Reflection).

[0008] Document EP 3 422 086 discloses the use of electrochromic lenses to adjust the color based on the environment characteristics. However, this document focuses on solutions to propose vision improvement for the user, such as better contrast, better color vision... and does not address an observer point of view and so the aesthetic of the wearer.

[0009] Therefore, there is an unfulfilled need to help a wearer to select, among a large palette of available colors provided by an electrochromic lens, the color the most relevant for wearer’s appearance, considering both his / her current environment color characteristics, and his / her own color characteristics (such as hair, dress code, make-up, frame color...)

[0010] SUMMARY OF THE DISCLOSURE

[0011] To solve this problem, the disclosure provides a method implemented by computer means for adjusting color of at least one active lens of a smart spectacle when used by a wearer according to claim 1 , said method comprising:

[0012] - acquiring at least one image representing said wearer and its current environment;

[0013] - determining a first set of N colors, N being an integer greater or equal to 1 , said first set of N colors comprising dominant colors in said at least one acquired image;

[0014] - determining a second set of M harmonized colors, M being an integer greater or equal to 1 , based on said first set of N colors;

[0015] - selecting at least one harmonized color within said second set of M harmonized colors; and

[0016] - controlling said at least one active lens so that at least one external color of said at least active lens corresponds to the selected at least one harmonized color.

[0017] In some embodiments, acquiring at least one image may comprise capturing said at least one image representing said wearer and its current environment.

[0018] In other embodiments, acquiring at least one image may comprise receiving said at least one image representing said wearer and its current environment.

[0019] In some embodiments, determining said first set of N colors may comprise extracting a predetermined number of dominant colors from said at least one acquired image. In such embodiments, determining said first set of N colors may further comprise selecting at least one zone of interest within said at least one acquired image, and said predetermined number of dominant colors is extracted from the at least one selected zone of interest.

[0020] In such embodiments, said predetermined number of dominant colors is greater than two and is preferably equal to six.

[0021] In such embodiments, the method may further comprise completing the first set of N colors by at least one additional color selected by the wearer.

[0022] If said first set is limited to a unique color, determining said second set of harmonized colors comprises calculating a harmonized color corresponding either to the complementary color of said unique color, or to a monochromatic color with the same hue as for the unique color, but a difference of chroma and luminance, or to an analogous harmony color.

[0023] If said first set is limited to a pair of colors, determining said second set of harmonized colors comprises calculating harmonized colors each corresponding either to the complementary color of a respective color of the pair, or to a monochromatic color with the same hue as for the respective color, but a difference of chroma and luminance, or to an analogous harmony color, or to a triadic harmonious color.

[0024] If said first set comprises three or more colors, determining said second set of harmonized colors comprises using color harmonic templates on a hue wheel, such as i type, V type, L type, I type, T type, Y type or X type templates.

[0025] In some embodiments, determining said second set of harmonized colors may comprise canceling at least one determined color based on lens constraints.

[0026] In some embodiments, selecting at least one harmonized color may comprise generating a color harmony score between each of the M harmonized colors within said second set and a predetermined color.

[0027] In some embodiments, generating a color harmony score may comprise applying a color harmony model for two colors. In other embodiments, generating a color harmony score may comprise determining arc-length distances on a hue wheel. In some embodiments, generating a color harmony score may comprise incorporating each of the M harmonized colors within chosen image and calculating a color harmony model based on the analysis of each pixel.

[0028] In some embodiments, selecting at least one harmonized color may comprise having the wearer select among a final set of harmonized colors having the highest scores.

[0029] The present disclosure also provides an ophthalmic system according to claim 12, comprising a smart spectacle equipped with at least one active lens and means configured for:

[0030] - acquiring at least one image representing said wearer and its current environment;

[0031] - determining a first set of N colors, N being an integer greater or equal to 1 , said first set of N colors comprising dominant colors in said at least one acquired image;

[0032] - determining a second set of M harmonized colors, M being an integer greater or equal to 1 , based on said first set of N colors;

[0033] - selecting at least one harmonized color within said second set of M harmonized colors; and

[0034] - controlling said at least one active lens so that at least one external color of said at least active lens corresponds to the selected at least one harmonized color.

[0035] In some embodiments, said means are fully integrated in said smart spectacle.

[0036] In other embodiments:

[0037] - said smart spectacle further comprises wireless communication means;

[0038] - said ophthalmic system further comprises at least one distinct equipment designed for communicating with said wireless communication means according to a least one wireless protocol; and

[0039] - said means are distributed between said smart spectacle and said at least one distinct equipment. Said at least one distinct equipment maybe a server, or a smart phone, or a smart watch, or a personal computer, and said at least one wireless protocol is Bluetooth or WiFi or a mobile telecommunications standard.

[0040] BRIEF DESCRIPTION OF THE DRAWINGS

[0041] For a more complete understanding of the description provided herein and the advantages thereof, reference is now made to the brief descriptions below, taken in connection with the accompanying drawings and detailed description, wherein like reference numerals represent like parts.

[0042] FIG. 1 a shows an example of an ophthalmic system according to one possible embodiment of the present disclosure;

[0043] FIG. 1 b shows an example of an ophthalmic system according to another possible embodiment of the present disclosure;

[0044] FIG. 1 c shows an example of an ophthalmic system according to another possible embodiment of the present disclosure;

[0045] FIG. 1 d shows an example of an ophthalmic system according to one possible embodiment of the present disclosure;

[0046] FIG. 2 is a flow chart illustrating different steps of a method implemented by computer means for adjusting color, according to one possible embodiment;

[0047] FIG. 3 is a flowchart illustrating different successive sub-steps which may be performed for determining a first set of N dominant colors based on acquired image(s).

[0048] FIG. 4 shows an example of an original acquired image, areas of this original image when divided and the main colors of these areas.

[0049] FIG. 5 is a flowchart illustrating different successive sub-steps which may be performed for selecting a harmonized color, according to possible embodiments of the invention.

[0050] DETAILED DESCRIPTION OF THE DISCLOSURE

[0051] In the description which follows, the drawing figures are not necessarily to scale and certain features may be shown in generalized or schematic form in the interest of clarity and conciseness or for informational purposes. In addition, although making and using various embodiments are discussed in detail below, it should be appreciated that as described herein are provided many inventive concepts that may embodied in a wide variety of contexts. Embodiments discussed herein are merely representative and do not limit the scope of the disclosure. It will also be obvious to one skilled in the art that all the technical features that are defined relative to a process can be transposed, individually or in combination, to a device and conversely, all the technical features relative to a device can be transposed, individually or in combination, to a process and the technical features of the different embodiments may be exchanged or combined with the features of other embodiments.

[0052] The terms “comprise” (and any grammatical variation thereof, such as “comprises” and “comprising”), “have” (and any grammatical variation thereof, such as “has” and “having”), “contain” (and any grammatical variation thereof, such as “contains” and “containing”), and “include” (and any grammatical variation thereof such as “includes” and “including”) are open-ended linking verbs. They are used to specify the presence of stated features, integers, steps or components or groups thereof, but do not preclude the presence or addition of one or more other features, integers, steps or components or groups thereof. As a result, a method, or a step in a method, that “comprises”, “has”, “contains”, or “includes” one or more steps or elements possesses those one or more steps or elements, but is not limited to possessing only those one or more steps or elements.

[0053] Figure 1 a is an example of an ophthalmic system according to one possible embodiment of the present invention. The ophthalmic system comprises a smart spectacle 1 including a frame 10 equipped with one active lens 1 1 .

[0054] The wording “active lens” means electrochromic lens, which tint can be changed by applying a corresponding control signal.

[0055] In some embodiments, frame 10 can be equipped with two active lenses, one for each wearer’s eye.

[0056] Smart spectacle 1 also comprises a battery 13, a memory 14, a central processing unit 15, a sensor 16 and a command 12 to generate the control signal and command an external color of active lens 1 1. Sensor 16 can capture images representing the wearer and his current environment. Figure 1 b is another example of an ophthalmic system according to a possible embodiment of the present invention. The ophthalmic system comprises a smart spectacle 1 similar to smart spectacle of figure 1 a, except that smart spectacle 1 further includes here a wireless communication module 17a. The ophthalmic system further comprises one or several servers 2 for communicating with wireless communication module 17a according to any mobile communications standard. The ophthalmic system can capture images representing the wearer and his current environment through sensor 16, or acquire such images from the server 2 through wireless communication module 17a.

[0057] Figure 1 c is another example of an ophthalmic system according to a possible embodiment of the present invention. The ophthalmic system comprises a smart spectacle 1 similar to smart spectacle of figure 1 a, except that smart spectacle 1 further includes here two wireless local area communication modules such as WiFi module 17b, and a Bluetooth module 17c. Such modules 17b and 17c enable smart spectacle 1 to communicate wirelessly with external distinct equipments 3a, 3b and 3c of the ophthalmic system. Distinct equipments 3a, 3b and 3c respectively represent a smart phone, a connected watch, and a personal computer. Such distinct equipments 3a, 3b and 3c may be equipped with sensors or cameras. In such cases, sensors embedded in any distinct equipments 3a, 3b and / or 3c may be able to capture images representing the wearer and his current environment.

[0058] Figure 1 d is another example of an ophthalmic system according to a possible embodiment in the invention. Ophthalmic system of figure 1 d is a mix of ophthalmic system of figure 1 b and ophthalmic system of figure 1 c. Here, smart spectacle 1 includes the above-mentioned wireless communication module 17a, WiFi module 17b and Bluetooth module 17c. Smart spectacle 1 is thus able to communicate with several distinct equipments included in the system, such as one or several servers 2 of the ophthalmic system, a smartphone 3a, a connected watch 3b and a personal computer 3c.

[0059] All the above-described embodiments have in common that the optical system according to the invention has means, to acquire at least one image representing said wearer and his current environment, and to process said at least one image in order to be able to control active lens 1 1 so that an external color of the lens corresponds to a harmonized color, as will be explained below.

[0060] Said means are fully integrated in the smart spectacle 1 according to figure 1 a.

[0061] Said means can also be distributed between smart spectacle 1 and at least one distinct equipment such a server(s) 2, smart phone 3a, smart watch 3b or personal computer 3c for the ophthalmic systems of figures 1 b, 1 c or 1 d.

[0062] Figure 2 is a flow chart illustrating the different steps of a method implemented by computer means for adjusting color of one or two active lenses of a smart spectacle when used by a wearer, in a possible embodiment of the present invention. Smart spectacle is any of the smart spectacles 1 described above with reference to figures 1 a to 1d.

[0063] At step 100, one or several images representing the wearer and its current environment are acquired. Each acquired image is then processed, at step 1 10, in order to determine dominant colors within the acquired images. Step 1 10 generates a first set of N colors, said first set comprising the determined dominant colors. In some embodiments, integer N can be equal to 1 . In other possible embodiments, integer N can be greater to 1 . At step 120, a second set of M harmonized colors is determined, based on the first set of N colors. Then, at step 130, a harmonized color is selected within the second set of M harmonized colors. At last, active lens or lenses are controlled at step 140, so that an external color of the lens(es) corresponds to the selected harmonized color.

[0064] Possible implementation of the different above steps 100 to 140 will be now further detailed:

[0065] Imaqe(s) Acquisition (step 100)

[0066] The first step 100 of the process consists in acquiring at least one image representing the wearer and its current environment.

[0067] Images representing at least the wearer can be provided by the wearer who has taken self-pictures which show color characteristics linked to the wearer (such as hair, dress code, make-up, frame color...). The same image can also represent the wearer’s current environment. Alternatively, other images showing only the current environment of the wearer can be separately acquired. For ophthalmic system of figures 1 a, 1 c and 1 d, the wearer wearing smart spectacle 1 can for instance capture, through sensor 16, a self-image while staying in front of a mirror and capture images of the environment. For ophthalmic system of figure 1 b, images showing only the environment of the wearer can have been previously captured and stored in a database of server 2 and acquired by smart spectacle 1 through wireless communication module 17a. For ophthalmic system of figures 1 c and 1d, self-pictures of the wearer and / or the environment can be captured by a sensor integrated within distinct equipments 3a, 3b and / or 3c.

[0068] The acquired images can be of the hyperspectral type or of the RGB type (ideally without any post-processing).

[0069] If the image is of the hyperspectral type, this image is first converted in RGB through a color appearance model (for instance the one named iCAM06).

[0070] This model is for instance described into “Kuang, J., Johnson, G. M., & Fairchild, M. (2007). iCAM06: A refined image appearance model for HDR image rendering. Journal of Visual Communication and Image Representation, 18(5). https: / / doi.Org / 10.1016 / j .jvcir.2007.06.003.

[0071] Determination of the first set of N colors (step 1 10)

[0072] Dominant colors may be determined according to the principles disclosed in “Automatic colour segmentation and colour palette identification of complex images”, Raza et aL, JAIC 2021 , ISSN 2227-1309, and in “Categorizing color shifts due to tinted glazing via dominant colors of the scene”, Raza et aL, CRA, volume 46, issue 3, pages 623-634, June 2021 , explained below.

[0073] Figure 3 is a flowchart illustrating different successive sub-steps which can be performed for implementing step 1 10 and determining a first set of N dominant colors, according to the invention, based on image(s) acquired at step 100.

[0074] The notion of dominant colors can be defined as the colors that are the most present in the image(s).

[0075] In order to identify a predetermined number of dominant or main colors on the image(s), said image(s) may be divided in a predetermined number of groups of areas (also called “clusters”) in each of which one of said dominant colors is representative of the colors of the areas of the cluster. We can first describe in detail how this operation may be performed when only one single image of the wearer within its current environment has been acquired.

[0076] Sub-step 1 1 1 consists in undoing any gamma correction on the acquired image. Such a gamma correction is generally applied on pictures to be faithfully displayed on a screen. Here, this correction is removed so as to obtain a raw image.

[0077] In a following sub-step 1 12, the RGB raw image is transformed into a new image coded in a determined color space. Thanks to this sub-step, the color of each pixel of the image can be transformed into a color space that enables an easier processing of the colors. The idea is to characterize the colors of the images by parameters easy to find (called hereinafter “components”). Here, the CIELAB color space is chosen over other color spaces not only because of simple, homogeneous, and uniform color distribution but also because of the perceived effectiveness of CIELAB color differences. In this color system, a color can be defined by three components: a clarity or lightness referenced L* (0 for black and 10 for white) and two coordinates referenced (a*, b*) or (H*, C*). The two components a*, b* characterize the color of each pixel and are expressed in Cartesian coordinates. The components H*, C* correspond to these components a*, b*, expressed in polar coordinates. They are named hue H* and chroma C* (that relates to the pep of the color, and more precisely to its saturation or contrast or vibrance or vividness, and that lies between a 0 for a low saturation and 100 for a high saturation). In this CIELAB system, the differences of value AL*, hue AH* and chroma AC* are standardized and easy to interpret.

[0078] But the CIELAB color space requires the knowledge of the white point of an illuminant, which can create a problem for images with unknown illuminants.

[0079] Consequently, in a sub-step 1 12, the processing unit first estimates this illuminant. To this end, various illuminant estimation methods exist.

[0080] The White Patch Retinex algorithm and Gray world algorithm (Buchsbaum, 1980; Land, 1977) are the commonly used algorithm though they are prone to large estimation errors (Hordley 2006). Another method that exists identifies the bright and dark pixels of the image as per their distance from the average color of the scene. It then performs Principal Component Analysis (PCA) on the bright and dark pixels. The first component of the PCA is thus the estimated illuminant. This method is described in “Cheng, D., Prasad, D. K., & Brown, M. S. (2014). Illuminant estimation for color constancy: Why spatial-domain methods work and the role of the color distribution. Journal of the Optical Society of America A, 31 (5), 1049. https: / / doi.Org / 10.1364 / JOSAA.31 .001049”.

[0081] Another method is described in “Hordley, S. D. (2006). Scene illuminant estimation: Past, present, and future. Color Research & Application, 31 (4), SOS- 314. https: / / d0i.0rg / l 0.1002 / COL20226”.

[0082] Then, in a sub-step 1 13, thanks to the illuminant white-point, the linear image is converted into a LAB image (i.e., into an image characterized by the three components L*, a*, b* or L*, H*, C*).

[0083] Then, during a sub-step 114, this LAB image is filtered in order to obtain a blurred image easier to process. The blurring intensity is preferably adjusted to give less importance to the edges and local differences in the images while preserving the general distribution of colors in this image. The chosen filter is a low pass Gaussian filter, the blurring intensity of which being noted a. The blurring intensity a corresponds to the standard deviation of the distribution. It controls the variance around a mean value of the Gaussian distribution. Sub-step 1 14 advantageously accelerates the convergence of the algorithm used to find the main colors of the image. Indeed, blurring with a relatively high blurring intensity a gives less importance to local differences and brings out the global color tendency of the image. The blurred LAB image is then ready for a clustering process which aims at dividing the image in a predetermined number of groups of areas (the clusters) in each of which a main color dominates.

[0084] The idea of the clustering process consists indeed in distributing the colors present in the image into various clusters of similar colors and retrieve the dominant colors of the scene. To this end, at sub-step 1 15, the processing unit uses a K- means algorithm or an algorithm deriving from it. Here a K-means-H- algorithm is used for segmenting the image in a predetermined number of clusters (as described in Arthur & Vassilvitskii, 2007). This predetermined number is at least equal to two. It is preferably greater than three and it is here equal to six. In other words, the algorithm consists in dividing the images in distinct areas each associating to one of these six colors. This algorithm will not be described in detail here, but we can provide the following explanations.

[0085] The K-means algorithm locates randomly six initial seeds and works in loops to try to divide the images in clusters.

[0086] The K-means algorithm determines all the seeds randomly. This leads to the initialization of far-away centroids leading to poor and lengthy clustering.

[0087] Here, the K-means++ algorithm determines the first seed by random assignment, but the rest of the seeds are carefully determined to maximize the distance between the centroids. This approach takes longer in initializing, but the clustering process has been proved to be faster than the original K-means clustering, thus globally reducing the time taken to converge.

[0088] This algorithm is described in “Aubaidan. (2014). Comparative study of k- means and k-means-H- clustering algorithms on crime domain. Journal of Computer Science, 10(7), 1 197-1206.

[0089] This algorithm results on the determination of six clusters. Each cluster corresponds to a group of areas where a main color dominates.

[0090] Consequently, the resulting six clusters enable to identify the spatial location of the untouched color clusters and for instance to calculate the distribution of each cluster (in percentage of pixels).

[0091] Knowing the distribution of each cluster on the blurred image, a median sRGB triplet corresponding to the main color of a cluster is calculated for each cluster, on the RGB image (or on a sRGB image that is an image coded in standard RGB). Then, this sRGB triplet is converted into another triplet formed by the three components L*, a*, b* or L*, H*, C*.

[0092] During a further sub-step 1 16, six “close main colors” can be identified, i.e., six colors that are very next from the six main colors and that are well defined in a dictionary (or “palette”) registering a finite number of colors. To this end, each found main color are compared with predefined colors registered in the dictionary. Then, each found main color is approximated by the predefined color that comes closest to it (named “standard main color”). More specifically, the triplet of each found main color is compared with standard data of a color system. Here, the color system is preferably the Munsell color system, which fits perfectly with the CIELAB color space. More precisely, the triplet is compared with the ISCC-NBS color dictionary in order to try to find, for each found main color, the closest color triplet (and its name) as defined through the CIEDE2000 formula (Cobeldick, 2019; Judd & Kelly, 10 1939; Sharma et aL, 2005). We can note here that this dictionary uses 13 basic color names for the first level, and that 29 intermediate color categories form a finer second level while 20 adjectives like vivid, dull, bright, moderate etc. form the finest third level. Finally, the final dictionary contains 267 distinct color names. At this stage, the names of the six standard main colors of the image are identified. The process can then check whether two clusters or more are associated to the same color. A redundancy can indeed appear if two distinct found main colors are approximated by the same standard color. If any, the two clusters are merged to form a new cluster, and the proportion in percentage of the color distribution is updated.

[0093] Figure 4 shows an example of acquired RGB image 40. This image does not represent the wearer in its current environment but is shown to clearly illustrate the clustering operation of figure 3. On the right of this image 40 are shown the six clusters 41 - 46 obtained by means of the clustering operation of figure 3. It appears that each cluster corresponds to a group of areas having similar colors. Under this image 40 is represented a bar 47 illustrating the percentage of distribution of each main color. On the right side, a box 48 gives the name of the six standard main color. Finally, a color circle 49 illustrates the coordinates, in the a*, b* system, of these standard main colors.

[0094] The clustering operation was described with a single image.

[0095] In some embodiments, several distinct images typical of the wearer and its current environment may have been acquired. The clustering operation can then be performed slightly differently.

[0096] The six standard main colors may indeed be identified as a function of all the acquired images. The acquired images can be associated into a single big image (not by superimposing the images but by aggregating them). They can be associated side to side, or the one above the other or in a matrix shape. Then, the sub-steps 1 1 1 to 1 16 can be performed on this big image. In a variant, sub-steps 1 1 1 to 1 16 can be performed for each acquired image, in order to find for each image six clusters associated to six “intermediate standard main colors”. Then, the final six standard main colors are calculated according to the proportion of each intermediate standard main color on said distinct images. For instance, these final six standard main colors may be the ones that are the most represented on the images.

[0097] Here, we can note that an analysis made on a single image and an analysis made on a plurality of images produce different dominant colors.

[0098] In another embodiment, at least one zone of interest within the acquired image(s) can be identified, i.e., by the wearer, and step 1 10 is performed on the selected zone of interest.

[0099] In another embodiment, the set of N colors can be completed by at least one additional color selected by the wearer.

[0100] Determination of the second set of M harmonized colors (step 120)

[0101] The way according to which the M harmonized colors are determined depends on the number N of colors determined in the first set at step 1 10.

[0102] In one embodiment in which N is equal to 1 , which means that the first set is limited to a unique dominant color, the corresponding harmonized color calculated at step 120 can correspond to a complementary color, i.e., to a color which is opposite to the unique dominant color on a color hue wheel. For instance, in case the unique dominant color is represented by a triplet [Ra, Ga, Ba] and the RGB space, the complementary color determined at step 120 as being in harmony with the unique dominant color is represented by the triplet [255-Ra, 255- Ga, 255-Ba].

[0103] In another embodiment in which N is equal to 1 , the corresponding harmonized color calculated at step 120 can correspond to a monochromatic harmony color, i.e., to a color with the same hue than the dominant color, but with a difference of chroma and luminance. For instance, in case the unique dominant color is represented by a triplet [Ra, Ga, Ba] and the RGB space, the corresponding monochromatic harmony color determined at step 120 as being in harmony with the unique dominant color is represented by the triplet [Ra + a, Ga + a, Ba + a], where a is a predetermined constant. In another embodiment in which N is equal to 1 , the corresponding harmonized colors calculated at step 120 can correspond to an analogous harmony color, i.e., to a color with a small hue difference with the hue of the dominant color. For instance, in case the unique dominant color is color A represented in HSL color space, the corresponding analogous harmony colors determined at step 120 as being in harmony with the unique dominant color are color B and C such that

[0104] Color B = Color A + 30° and

[0105] Color C = Color A - 30°.

[0106] In one embodiment in which N is equal to 2, which means that the first set is composed of a pair of dominant colors, the corresponding harmonized colors may also correspond to the complementary colors or the monochromatic harmony colors of each of the dominant colors as defined above.

[0107] In another embodiment in which N is equal to 2, the hue, chroma and luminance of each of the dominant colors can be determined and compared. In case the colors of the pair have identical chroma, identical luminance, but small hue difference (i.e., when hue keeps within each cluster inferior to + / - 60° or remain under color boundaries), one analogous harmonized color is chosen at step 120 with identical chroma and a difference of hue with one color of the pair that matches the hue difference of the dominant colors. For instance, in case the color A and color B, represented in HSL color space, corresponds to the pair of dominant colors, one corresponding analogous harmony color C determined at step 120 as being in harmony with the pair of dominant colors A and B can be calculated using the formula:

[0108] Color C = Color A + AH, where AH is the hue difference between color A and B.

[0109] In case the colors of the pair have identical chroma, identical luminance, but high hue difference (i.e., when hue is superior to + / - 60° or exceed color boundaries, one triadic harmonized color is chosen at step 120 with identical chroma and a difference of hue with each color of the pair that matches the hue difference of the dominant colors. Triadic colors are defined as colors that are evenly spaced around the color wheel In another embodiment in which N is equal to 3, the hue, chroma and luminance of each of the dominant colors can be determined and compared. In case the dominant colors have identical chroma and identical luminance, one corresponding tetradic harmony color is determined at step 120 as being in harmony with the trio of dominant colors.

[0110] Other known geometric rules for color harmony in a color wheel, such as split-complementary rule or square rule, can be applied.

[0111] In case the integer N is greater than 3, and especially for a large number of colors in the first set of dominant colors, it may be advantageous to use other color harmonic templates on the hue wheel, such as i type, V type, L type, I type, T type, Y type or X type templates as defined in "Color Harmonization," by Daniel Cohen- Or et aL, ACM Transactions on Graphics (TOG), Volume 25:3. July 2006. Proceedings of ACM SIGGRAPH 2006, pp. 624 - 630 © 2006 ACM, Inc, or new versions developed by Chistel Chamaret.

[0112] Additive constraints can suppress some combinations, such as dyes availability.

[0113] As an option, color harmonic rules can be oriented to match a color temperature according to the time of day, to respect circadian rhythm. It can consider the optimal Tv deduced from luminance and indoor / outdoor analyses.

[0114] Selection of a harmonized color (step 130)

[0115] Figure 5 is a flowchart illustrating different successive sub-steps which may be performed for implementing step 130 and selecting a harmonized color, according to possible embodiments of the invention, within the second set of M harmonized colors generated at step 120.

[0116] A first sub-step 131 consists in generating a color harmony score between each of the M harmonized colors within the second set and a predetermined color.

[0117] In some embodiments, the predetermined color can correspond to the wearer preferred color, or to the wearer skin tone, or to a trend color or seasonal trend, or to a wearer friend’s preferred color.

[0118] In one embodiment, the predetermined color corresponds to an average of the N colors belonging to the first set. In other embodiments, a color harmony score is generated between each of the M harmonized colors within the second set and the entire sets of colors of an image.

[0119] To calculate each color harmony score, one embodiment uses the two-color harmony model CH disclosed in “A color harmony model for two-color combinations”, Ou, L., and Luo, M. R. (2006). Color Research & Application, 31 , 191-204 according to the following formula:

[0120] CH = He + HL + HH where He is a chromatic effect factor, HL is a lightness effect factor, and HH is a hue effect factor, and where

[0121] He = 0.04 + 0.53 * tanh (0.8 - 0.045 AC)

[0122] HL = HLsum + HAL

[0123] HLsum = 0.28 + 0.54*tanh (- 3.88 + 0.029Lsum) in which Lsum = L-| + L2

[0124] HAL = 0.14 + 0.15*tanh (-2 + 0.2AL) in which AL = |L L2||

[0125] HH = HSYI + HSY2

[0126] HSY = EC*(HS+EY)

[0127] Ec = 0.5 + 0.5*tanh(-2 + 0.5*b)

[0128] Hs = - 0.08 - 0.14*sin(hab + 50°) - 0.07*sin (2hab + 90°)

[0129] EY = [(0.22L* - 12.8) / 10] exp(( 90° - hab) / 10 - exp[( 90° - hab) / 10]

[0130] The larger the CH value, the higher the color harmony score.

[0131] The process is replicated for each color combination composed of one color from the M set and the predetermined color.

[0132] In another embodiment adapted to the case in which the captured image comprises the wearer wearing the spectacle with lens and his environment, it is possible, for each harmonized color within the second set of M harmonized color: - to process the acquired image in order to generate a modified image in which the color of the lens has been replaced with the harmonized color;

[0133] - then to calculate color harmony distance and maps according to the teaching of Christel Chamaret et al in document entitled “Harmony-Guided Quality Assessment”, Computer Vision and Pattern Recognition Workshops (CVPRW),2013 IEEE Conference on , vol., no., pp.961 ,967, 23-28 June 2013.

[0134] More particularly, according to this teaching, for a given harmonic template

[0135] Tmon the hue wheel, such as i type, V type, L type, I type, T type, Y type or X type templates, a hue h is considered harmonious if it is enclosed by a sector (meaning its harmonious distance is 0), while a hue outside the sector is not harmonious regarding a certain proportion defined by the hue distance dm(h), which is evaluated by computing the arc-length distance on the hue wheel (measured in degrees) to the closest sector according to the following equation: where 1. 1 is the arc-length distance and [. ] = max(0; .).

[0136] Then, assuming that each template (associated with its optimal angle) provides harmony information about the picture, dm maps are computed for all templates and combined at the pixel level. At each pixel u = (x; y) with associated hue h(u), a harmony distance map G(u) accumulates the harmony distances according to the following equation, in which the contribution of each template is weighted according to its respective energy Emto give more importance to templates having low energy: where s and v are saturation and value of the image.

[0137] The above-process is replicated for each harmonized color within the second set of M harmonized color, to obtain the color harmony for each possible harmonized color. Whatever the process implemented to generate the M color harmony scores in first sub-step 131 , only one harmonized color must be selected at the end of step 130. In one embodiment, as shown in figure 5, the method comprises a sub-step 132 wherein the wearer can select the harmonized color within a final sub set of harmonized color having the highest harmony scores. For instance, the harmonized colors having the three best scores can be proposed to the wearer via an application loaded in a mobile phone, for selection. In another embodiment, best scored M colors could be showcased to the wearer via the smart spectacle and a live demo switching the best scored harmonized colors every 5 seconds. The wearer could then touch the frame to choose the harmonized color he wants.

[0138] In another embodiment, the ophthalmic system can automatically select the final harmonized color which has the highest harmony score.

[0139] Active lens(s) control (step 140)

[0140] As indicated above, once the final harmonized color has been selected (either automatically, or by the wearer), active lens 1 1 is controlled at step 140, so that an external color of the lens corresponds to the selected harmonized color.

[0141] In some embodiments, active lens 1 1 is a transmissive electrochromic lens comprising several cells between two transparent supports with a transparent electrode. Each cell comprises electrochromic dye compounds having different oxidation potentials. The cells are driven independently using a command signal including a specific driving voltage per cell. It is thus possible to have a range of colors addressing any combination of colors of the cells.

[0142] In other embodiments, active lens 1 1 is a reflective electrochromic lens of the type disclosed in US 2018 239 170 using liquid crystal in Cholesteric phase, having pitch determined according to the wavelength to be filtered. Combining multiple cells with different pitches allows to have variable colors by reflection

[0143] As a possible example, a parameter table giving chromatic coordinates (CIEL*a*b* deduced from CIEXYZ) for different possible combined values for driving voltages for the cells can be used to determine what voltage to apply to each cell depending on the final external color wished (the selected harmonized color). These chromatic coordinates correspond to the observed colors, from an observer point of view.

[0144] Practically, for a transmissive electrochromic lens, these chromatic coordinates can be determined using a white background, a colorimeter, and EC lenses in between. As an alternative, we may use as a source different type of illuminants (D65 for instance) in front of the lenses, and replace the background behind the lenses by a diffuser having similar spectral signature compared to the skin (similar reflectance for different visible wavelength)

[0145] For a reflective electrochromic lens, these chromatic coordinates can be directly determined from a colorimeter measuring the reflection on the lens of a white background.

[0146] The parameter table may be recorded in memory 14 of smart spectacle 1 of figure 1 a, or in a memory of the servers 2 or of the distinct equipments 3a-3c for the ophthalmic system according to figures 1 b to 1 d.

[0147] As already disclosed above, each of the ophtlamic systems described in reference to figures 1 a to 1 d has means to perform steps 100-140 of figure 2. Hence:

[0148] For the ophthalmic system according to figure 1 a, said means are fully integrated in the smart spectacle 1 , and include sensor 16 for acquiring the images (step 100), memory 14 and central processing unit 15 with corresponding software instructions implementing steps 110 to 130, and command 12 to control the active lens so that an external color of the lens corresponds to the final selected harmonized color (step 140).

[0149] For the ophthalmic systems of figures 1 b, 1 c or 1 d., said means can also be distributed between smart spectacle 1 and at least one distinct equipment such a server(s) 2, smart phone 3a, smart watch 3b or personal computer 3c.

[0150] Although representative embodiments have been described in detail herein, those skilled in the art will recognize that various substitutions and modifications may be made without departing from the scope of what is described and defined by the appended claims.

Claims

CLAIMS1 . A method implemented by computer means for adjusting color of at least one active lens (1 1 ) of a smart spectacle (1 ) when used by a wearer, said method comprising:- acquiring (100) at least one image representing said wearer and its current environment;- determining (1 10) a first set of N colors, N being an integer greater or equal to 1 , said first set of N colors comprising dominant colors in said at least one acquired image;- determining (120) a second set of M harmonized colors, M being an integer greater or equal to 1 , based on said first set of N colors;- selecting (130) at least one harmonized color within said second set of M harmonized colors; and- controlling (140) said at least one active lens (1 1 ) so that at least one external color of said at least active lens corresponds to the selected at least one harmonized color, wherein:- if said first set is limited to a unique color, determining (120) said second set of harmonized colors comprises calculating a harmonized color corresponding either to the complementary color of said unique color, or to a monochromatic color with the same hue as for the unique color, but a difference of chroma and luminance, or to an analogous harmony color;- if said first set is limited to a pair of colors, determining (120) said second set of harmonized colors comprises calculating harmonized colors each corresponding either to the complementary color of a respective color of the pair, or to a monochromatic color with the same hue as for the respective color, but a difference of chroma and luminance, or to an analogous harmony color, or to a triadic harmonious color; and- if said first set comprises three or more colors, determining (120) said second set of harmonized colors comprises using color harmonic templates on a hue wheel, such as i type, V type, L type, I type, T type, Y type or X type templates.

2. A method according to claim 1 , wherein acquiring (100) at least one image comprises capturing said at least one image representing said wearer and its current environment.

3. A method according to claim 1 , wherein acquiring (100) at least one image comprises receiving said at least one image representing said wearer and its current environment.

4. A method according to anyone of the preceding claims, wherein determining (1 10) said first set of N colors comprises extracting (1 16) a predetermined number of dominant colors from said at least one acquired image.

5. A method according to claim 4, wherein determining (1 10) said first set of N colors further comprises selecting at least one zone of interest within said at least one acquired image, and wherein said predetermined number of dominant colors is extracted from the at least one selected zone of interest.

6. A method according to anyone of claims 4 or 5, wherein said predetermined number of dominant colors is greater than two and is preferably equal to six.

7. A method according to anyone of claims 4 to 6, further comprising completing the first set of N colors by at least one additional color selected by the wearer.

8. A method according to any of the preceding claims, wherein determining (120) said second set of harmonized colors comprises canceling at least one determined color based on lens constraints.

9. A method according to any of the preceding claims, wherein selecting (130) a harmonized color comprises generating (131 ) a color harmony score between each of the M harmonized colors within said second set and a predetermined color.

10. A method according to claim 9, wherein generating (131 ) a color harmony score comprises applying a color harmony model for two colors, or determining arclength distances on a hue wheel.1 1. A method according to anyone of claims 9 and 10, wherein selecting (130) at least one harmonized color comprises having (132) the wearer select among a final set of harmonized colors having the highest scores.

12. An ophthalmic system comprising a smart spectacle (1 ) equipped with at least one active lens and means for:- acquiring (100) at least one image representing said wearer and its current environment;- determining (1 10) a first set of N colors, N being an integer greater or equal to 1 , said first set of N colors comprising dominant colors in said at least one acquired image;- determining (120) a second set of M harmonized colors, M being an integer greater or equal to 1 , based on said first set of N colors;- selecting (130) at least one harmonized color within said second set of M harmonized colors; and- controlling (140) said at least one active lens (1 1 ) so that at least one external color of said at least active lens corresponds to the selected at least one harmonized color wherein:- if said first set is limited to a unique color, determining (120) said second set of harmonized colors comprises calculating a harmonized color corresponding either to the complementary color of said unique color, or to a monochromatic color with the same hue as for the unique color, but a difference of chroma and luminance, or to an analogous harmony color;- if said first set is limited to a pair of colors, determining (120) said second set of harmonized colors comprises calculating harmonized colors each corresponding either to the complementary color of a respective color of the pair, or to a monochromatic color with the same hue as for the respective color, but a difference of chroma and luminance, or to an analogous harmony color, or to a triadic harmonious color; and- if said first set comprises three or more colors, determining (120) said second set of harmonized colors comprises using color harmonic templates on ahue wheel, such as i type, V type, L type, I type, T type, Y type or X type templates.

13. An ophthalmic system according to claim 12, wherein said means are fully integrated in said smart spectacle (1 ).

14. An ophthalmic system according to claim 12, wherein: - said smart spectacle (1 ) further comprises wireless communication means(17a-17c);- said ophthalmic system further comprises at least one distinct equipment (2, 3a-3c) designed for communicating with said wireless communication means (17a-17c) according to a least one wireless protocol, said at least one distinct equipment being a server (2), or a smart phone (3a), or a smart watch (3b), or a personal computer (3c), and wherein said at least one wireless protocol is Bluetooth or WiFi or a mobile telecommunications standard; and- said means are distributed between said smart spectacle (1 ) and said at least one distinct equipment (2, 3a-3c).

Images