Method and system for predicting the appearance of an object coated with at least one colored coating layer under different irradiation conditions.

Abstract

This invention provides a method and system for predicting the visual impression of colors, particularly automotive colors, without physically setting up different illumination conditions. [Solution] The computer system 200 includes a display device 202 coupled to databases 216, 218, and 220 via communication interfaces 234, 236, and 238. The display device 202 is coupled to sensors 222, 224, and 226 via communication interfaces 240, 242, and 244 and includes a screen 204 for displaying to the user, in particular via a graphical user interface (GUI), a digital representation of a provided colored coating layer, a digital representation of provided illumination conditions, and generated color data of the colored coating layer received from a processor 212.

Description

[Technical Field]

[0001] Embodiments described herein generally relate to methods and systems for predicting the appearance of an object coated with at least one colored coating layer under different irradiation conditions. More specifically, embodiments described herein relate to displaying the appearance of a coated object using a display device by predicting the appearance of the coated object under irradiation conditions selected by a user and displaying the predicted appearance of the coated object. [Background technology]

[0002] A surface reflects incident light to the observer's eye, and its reflective and absorptive properties create the visual impression of an object, thus giving it an appearance. For example, a surface that reflects all light rays appears white to the observer, while a surface that absorbs all light rays appears black. All other colors perceived by humans are due to the combination of light rays reflected and absorbed by an object's surface.

[0003] The visual impression of an object can be altered, for example, by applying a colored coating layer to its surface. In this case, the visual impression is determined by the colored coating layer. Vehicles, especially land vehicles such as automobiles, motorcycles, and truck bodies, are typically treated with multiple coating layers to improve their appearance and protect them from corrosion, scratches, chipping, ultraviolet rays, acid rain, and other environmental conditions.

[0004] The aforementioned coating is generally a composite coating system that requires the application of multiple coating layers to achieve the effects described above. In the case of metal substrates, an electrocoating is typically applied to the substrate and cured. This electrocoating is then coated with a cured primer coating or an uncured colored first basecoat before a further colored basecoat and clearcoat or colored clearcoat is applied over an uncured or "wet" first basecoat and / or further colored basecoat. In the case of plastic substrates, a primer coating is applied to the substrate and cured before at least one further colored basecoat coating and clearcoat or an uncolored clearcoat is applied over an uncured or "wet" colored basecoat coating. The applied basecoat and clearcoat are then co-cured. Thus, such systems are often described as "wet-on-wet," "two coats / one bake," or "three coats / one bake." Drying steps that do not lead to complete curing can be used between coating applications. The visual appearance in terms of color is typically achieved by a colored basecoat coating containing coloring pigments and / or effect pigments.

[0005] However, the visual impression of a colored object, such as an object coated with at least one colored coating layer, can change under different illumination conditions. This is due to the fact that the spectral power distribution of the light source, i.e., the relative power of various wavelengths of the light source, can change, and the way in which light is reflected to the human observer's eye also changes. This affects the visual impression and therefore the color of the coated object as perceived by the human observer. Furthermore, the power of the light source can also affect the perceived color of the object it illuminates. Thus, color is perceived differently depending on the light source and the power of the light source used to illuminate the colored object. For example, natural light can vary greatly depending on the weather, season, time of day, the position of the sun in the sky, the position of the object, and the surroundings of the object. Therefore, the color of a colored object perceived under clear skies may differ from the color of the same colored object perceived under cloudy skies.

[0006] The overall visual impression of a car is a crucial factor for consumers when making a purchase decision. Therefore, car designers and dealers typically enhance the visual appeal by using specially designed lighting conditions, such as artificial light sources with defined power and wavelength spectra, including fluorescent, incandescent, halogen, and LED lights. However, the degree to which lighting conditions affect color perception is highly color-dependent, so selecting appropriate lighting conditions often requires repeated testing of different conditions to find the optimal solution. Furthermore, since cars of various colors are typically displayed in showrooms, the showroom lighting conditions must be selected to ensure that each displayed color looks appealing. Thus, selecting appropriate lighting conditions can be a time-consuming and costly task. In addition, there is a growing demand from customers to view cars under actual lighting conditions, such as outdoors or in a garage, to determine if the visual impression meets their expectations. However, this is not always possible, as car dealers do not always have demo cars available in the desired colors. [Overview of the project] [Problems that the invention aims to solve]

[0007] Therefore, it is desirable to provide computer-based methods and systems that enable users, such as automotive color designers or car dealers, to simulate or predict the visual impression of colors, particularly automotive colors, under various lighting conditions without physically setting up different lighting conditions to determine the optimal lighting conditions. Furthermore, it is desirable to provide computer-based methods and systems that enable customers to determine whether an object they wish to purchase, particularly a colored car, will have the desired visual impression under real-world conditions, i.e., under natural light at the customer's location, thereby ensuring that customers are not disappointed after purchase by the visual impression of the object, particularly a colored car, under actual lighting conditions. Finally, it is desirable to provide computer-based methods and systems that reduce the number of colored objects, particularly colored cars, that need to be presented by sellers to customers in order to provide an overview of the available colors before sale. definition "Appearance" refers to the visual impression of a coated object to the observer's eye, including the perception of the spectral and geometric aspects of the surface integrated with its illumination and observation environment. Generally, appearance includes color, visual texture such as roughness caused by effect pigments, gloss, or other visual effects of the surface, particularly when viewed at different viewing angles and / or illumination angles.

[0008] "Digital representation" may refer to a computer-readable representation of the colored coating layer and illumination conditions. In particular, the digital representation of the colored coating layer includes an optimized bidirectional texture function (BTF) obtained by optimizing the initial BTF using captured spectral reflectance data, as described later. The digital representation of the colored coating layer may include further data, such as data on the formulation of the coating material used to prepare the colored coating layer, color code data, color name data, or a combination thereof. The data on the formulation of the coating material used to prepare the colored coating layer may include data on the types and amounts of at least some of the components present in the coating material. The digital representation of illumination conditions may include, for example, data indicating time, location, degree of sky haze (smog), data acquired from at least one illumination sensor and / or azimuth sensor and / or vantage point sensor, at least one high dynamic range (HDR) environment map, and a combination thereof. The environment map may be a predefined HDR environment map stored in a database, for example.

[0009] "Display device" refers to an output device for presenting information in a visual or tactile format (the latter may be used in tactile electronic display devices for the visually impaired). "Screen of display device" refers to the physical screen of the display device and the projection area of ​​a projection display device, etc.

[0010] An "interaction element" can refer to an element that is configured to receive user input.

[0011] A “communication interface” may refer to a software and / or hardware interface for establishing communication, such as the transfer or exchange of signals or data. A software interface is, for example, a function call or an API. A communication interface may include transceivers and / or receivers. Communication may be wired or wireless. A communication interface may be based on or support one or more communication protocols. Communication protocols may be short-range communication protocols such as Bluetooth® or WiFi, or wireless protocols such as long-range communication protocols such as cellular or mobile networks such as 2G, 3G, 4G, Long-Term Evolution (LTE), or 5G. Alternatively, in addition to these, a communication interface may be based on its own short-range or long-range protocol. A communication interface may support any one or more standard protocols and / or proprietary protocols.

[0012] "Computer processor" means any logic circuit and / or, generally, a device configured to perform calculations or logical operations, which are used to perform basic operations of a computer or system. In particular, a processing unit or computer processor may be configured to process basic instructions that drive a computer or system. For example, a processing unit or computer processor may include at least one arithmetic logic unit ("ALU"), at least one floating-point unit ("FPU") such as a mathematical coprocessor or a numerical coprocessor, a number of registers, in particular registers configured to supply operands to the ALU and store the results of calculations, and memory such as L1 and L2 cache memories. In particular, a processing unit or computer processor may be a multi-core processor. Specifically, a processing unit or computer processor may be a central processing unit ("CPU") or may include a CPU. The processing means or computer processor may be a composite instruction set computing ("CISC") microprocessor, a reduced instruction set computing ("RISC") microprocessor, a very long instruction word ("VLIW") microprocessor, or a processor implementing another instruction set, or a processor implementing a combination of instruction sets. The processing means may also be one or more purpose-specific processing devices, such as an application-specific integrated circuit ("ASIC"), a field-programmable gate array ("FPGA"), a composite programmable logic device ("CPLD"), a digital signal processor ("DSP"), or a network processor. The methods, systems, and devices described herein may be implemented as software within a DSP, a microcontroller, or any other side processor, or as hardware circuitry within an ASIC, CPLD, or FPGA. The term processing means or processor may also refer to one or more processing devices, such as a distributed system of processing devices deployed across multiple computer systems (e.g., cloud computing), and will be understood not to be limited to a single device unless otherwise specified. [Means for solving the problem]

[0013] To address the above problems, the following is proposed: a computer implementation method for predicting the appearance of an object coated with at least one colored coating layer and displaying the predicted appearance on a display device, the method comprising the following steps: (i) the step of providing a digital representation of the colored coating layer to a computer processor via a communication interface; (ii) - Displaying a graphical user interface containing multiple irradiation conditions on a display device; - The computer processor detects user input indicating that an irradiation condition has been selected from the displayed multiple irradiation conditions; - In response to the detected user input, the computer processor is used via the communication interface to obtain a digital representation of the irradiation conditions associated with the detected user input; The steps include providing a digital representation of the irradiation conditions; (iii) optionally providing the computer processor via the communication interface with a model derived from past irradiation conditions and / or at least one model derived from past environmental conditions; (iv) A step of generating color data for the colored coating layer using a computer processor based on a digital representation of the provided colored coating layer, a digital representation of the provided irradiation conditions, and optionally the provided model; (v) The step of displaying the generated color data received from the computer processor on the display device, Includes.

[0014] To address the above problems and to further resolve them, the following is proposed: a method for predicting the appearance of an object coated with at least one colored coating layer and displaying the predicted appearance on a display device, the method comprising the following steps: (i) Providing a digital representation of a colored coating layer to a computer processor via a communication interface, wherein the digital representation of the colored coating layer is provided by the following steps: - A step of determining the initial BTF of the colored coating layer using a camera-based measuring device; - A step of capturing spectral reflectance data of the colored coating layer for a predetermined number, i.e., a limited number of different measurement shapes, using a spectrophotometer; - To obtain the optimized BTF, the initial BTF in equation (1) below is used.

number

number

number

number

number

number

number

number

number

[0015] The essential advantage of the method and system according to the present invention is that it is possible to predict the visual impression of an object coated with at least one colored coating layer, particularly an automobile body, under different illumination conditions. Therefore, illumination designers can quickly select the illumination conditions necessary to enhance the attractiveness of the colored coated object without having to physically install and optimize various light sources. Furthermore, the method and system allows customers to quickly verify, before purchase, whether a colored object, particularly a colored automobile, will have the desired visual impression under real-world conditions, such as outdoors or indoors, for example, in a garage, thereby reducing potential disappointment after purchase. Finally, the method and system eliminates the need for sellers to provide objects of various colors so that customers can browse available colors and select their desired color before purchase, resulting in a reduction in display space and associated costs.

[0016] Further disclosure is made according to the first embodiment, A system for predicting the appearance of an object coated with at least one colored coating layer, wherein the system: - Optionally, a communication interface for providing a computer processor with a model derived from past irradiation conditions and / or at least one model derived from past environmental conditions; - At least one communication interface for providing a computer processor with a digital representation of the colored coating layer and a digital representation of the irradiation conditions; - A display device equipped with a screen; - Interaction elements for detecting user input; - Optionally, at least one illumination sensor and / or at least one orientation sensor adapted to sense the orientation of the display device and / or at least one viewpoint sensor adapted to sense the viewpoint of the user holding the display device; - A communication interface, a display device, and a processor that communicates with at least one of the illumination sensors and / or orientation sensors and / or viewpoint sensors, wherein the processor: • Receiving a digital representation of the colored coating layer via a communication interface; • Generate a user interface presentation containing multiple irradiation conditions, detect user input indicating that an irradiation condition has been selected from the displayed multiple irradiation conditions, and in response to the detected user input, retrieve a digital representation of the irradiation condition associated with the detected user input; Optionally, calculate the ambient illumination conditions surrounding the display device from the digital representation of the acquired illumination conditions and the received model; Based on the received digital representation of the colored coating layer and the received digital representation of the illumination conditions or the calculated ambient illumination conditions surrounding the display device, color data for the colored coating layer is generated. The processor is programmed to do so, Equipped with, The display device receives the generated user interface presentation and the generated color data for the colored coating layer from the processor, and displays the generated user interface presentation and color data. The digital representation of the colored coating layer includes an optimized bidirectional texture function (BTF) obtained by the steps described in relation to step (i) of the method of the present invention.

[0017] The processor may be the processor of the display device, that is, the processor may be located within the housing of the display device that houses the screen, or the processor may be located outside the display device, for example, within a further computing device such as a cloud computing device. This is preferable when the computing power of the display device's processor is not high enough to generate color data and optionally calculate ambient illumination conditions before generating the color data.

[0018] Further disclosures include alternative embodiments, A system for predicting the appearance of an object coated with at least one colored coating layer, wherein the system: - Optionally, a communication interface for providing a computer processor with a model derived from past irradiation conditions and / or at least one model derived from past environmental conditions; - At least one communication interface for providing a computer processor with a digital representation of the colored coating layer and a digital representation of the irradiation conditions; - A display device equipped with a screen; - Interaction elements for detecting user input; - Optionally, at least one illumination sensor and / or at least one orientation sensor adapted to sense the orientation of the display device and / or at least one viewpoint sensor adapted to sense the viewpoint of the user holding the display device; - A first processor that communicates with a communication interface, a display device, and at least one illumination sensor and / or orientation sensor and / or viewpoint sensor, wherein the first processor: • Generate a user interface presentation containing multiple irradiation conditions, detect user input indicating that an irradiation condition has been selected from the displayed multiple irradiation conditions, and in response to the detected user input, retrieve a digital representation of the irradiation condition associated with the detected user input; Optionally, calculate the ambient illumination conditions surrounding the display device from the digital representation of the acquired illumination conditions and the received model. The first processor is programmed to do so; - A communication interface and a second processor that communicates with the first processor, wherein the second processor is: • Receives a digital representation of the colored coating layer and a digital representation of the illumination conditions or ambient illumination conditions calculated by the first processor via a communication interface; Based on the received digital representation of the colored coating layer and the received digital representation of the illumination conditions or the calculated ambient illumination conditions received from the first processor, color data for the colored coating layer is generated. The second processor is programmed to do so, Equipped with, The display device receives color data generated from the colored coating layer from the processor and displays the generated color data. The digital representation of the colored coating layer includes an optimized bidirectional texture function (BTF) obtained by the steps described in relation to step (i) of the method of the present invention.

[0019] Further disclosures are made in yet another embodiment, A system for predicting the appearance of an object coated with at least one colored coating layer, the system comprising: a display; one or more computing nodes; and one or more computer-readable media having computer-executable instructions thereon, when executed by one or more computing nodes, structured to cause the system to perform the method of the present invention as described herein.

[0020] Further disclosures include: A non-transient computer-readable storage medium, which, when executed by a computer, includes instructions causing the computer to perform steps according to the computer implementation method described herein.

[0021] This disclosure applies equally to the systems, methods, computer programs, computer-readable non-transient storage media, and computer program products disclosed herein. Therefore, there is no distinction between systems, methods, computer programs, computer-readable non-transient storage media, and computer program products. All features disclosed in relation to the methods are also disclosed in relation to the systems, computer programs, computer-readable non-transient storage media, and computer program products disclosed herein.

[0022] Further disclosed is the use of a method or system disclosed herein for predicting the appearance of an object coated with at least one colored coating layer.

[0023] Further disclosed is an object coated with at least one coating layer, the color of which is predicted according to the method disclosed herein.

[0024] Embodiment At least one colored coating layer can be present on at least a portion of the object's surface. The presence of a colored coating layer on at least a portion of the object's surface is understood as follows: the colored coating layer is positioned on at least a portion of the object's surface, but does not need to be in direct contact with that portion. Therefore, other coating layers, such as the aforementioned cured electrocoat layer or primer layer, can be present between the colored coating layer and the object.

[0025] At least one colored coating layer may be a base coat layer or a colored clear coat layer. “Base coat layer” may refer to a cured color-impregnating intermediate coating layer commonly used in automotive and general industrial painting. The base coat material used to prepare the base coat layer may be formulated as a solid color (straight shade) or effect color coating. “Effect color coating” generally includes at least one effect pigment and optionally other coloring pigments or spheres that impart the desired color and effect. “Straight shade” or “solid color coating” primarily contains coloring pigments and does not exhibit a visible flop or two-tone metallic effect. The base coat layer is formed by applying the base coat material to a metal or plastic substrate, optionally including at least one cured coating layer, drying the applied base coat material, and curing the resulting base coat film. “Colored clear coat layer” may refer to a cured coating layer that is neither completely transparent and colorless like a clear coat nor completely opaque like a typical pigment base coat. Therefore, a colored clear coat layer may be transparent and colored, or translucent and colored. Coloring can be achieved by adding small amounts of pigment commonly used in base coat coating materials. A colored clear coat layer is formed by applying a colored clear coat material onto a common substrate containing at least one coating layer, such as a colored base coat layer, drying the applied colored clear coat material, and curing the resulting colored clear coat film.

[0026] The object may be an automobile or a part thereof. The term "automobile" refers to vehicles such as cars, vans, minivans, buses, SUVs (sports utility vehicles); trucks; semi-trucks; tractors; motorcycles; trailers; ATVs (all-terrain vehicles); pickup trucks; heavy-duty movers such as bulldozers, mobile cranes and earth movers; airplanes; boats; ships; and other means of transport that are commonly coated with at least one coating layer.

[0027] In one embodiment, the display device comprises a housing that contains the computer processor and screen used in steps (i) to (v). Thus, the display device comprises a computer processor and a screen. The housing may be made of plastic, metal, glass, or a combination thereof.

[0028] In another embodiment, the display device and computer processor that perform steps (i) to (iv) or steps (ii) to (iv) or step (iv) are configured as separate components. According to this embodiment, the display device comprises a housing that contains a screen, but does not include a computer processor that performs steps (i) to (iv) or steps (ii) to (iv) or step (iv) of the method of the present invention. Therefore, the computer processor that performs steps (i) to (iv) or steps (ii) to (iv) or step (iv) of the method of the present invention resides separately from the display device, for example, in a further computing device. The computer processor of the display device and the further computer processor are connected via a communication interface to enable data exchange. By using a further computer processor located outside the display device, higher computing power than that provided by the processor of the display device can be used, thus reducing the computation time required to perform these steps, and thus reducing the overall time until the generated color data is displayed on the screen of the display device. This allows the change in the appearance of a colored object to be displayed in real time or near real time when the illumination conditions are changed, and as a result, the user can intuitively and efficiently select the desired appearance, particularly the color, and / or the illumination conditions that produce the desired appearance. A further computer processor may be located on a server so that steps (i) to (iv) or steps (ii) to (iv) or step (iv) of the method of the present invention are performed in a cloud computing environment. In this case, the display device functions as a client device and is connected to the server via a network such as the Internet. "Client device" may mean a computer or program that, as part of its operation, relies on sending requests to the hardware or software of another program or computer to access services made available by the server. Preferably, the server is an HTTP server and is accessed via conventional Internet web-based technology.Internet-based systems are particularly useful when providing customers with a service to design the colors of colored coating layers manufactured from colored coating materials.

[0029] The display device may be either a mobile display device or a fixed display device, but a mobile display device is preferred. Fixed display devices include computer monitors, television screens, projectors, etc. Mobile display devices include laptops, or handheld devices such as smartphones and tablets.

[0030] The screen of the display device can be constructed according to any radiative or reflective display technology having an appropriate resolution and color gamut. An appropriate resolution is, for example, 72 dots per inch (dpi) or higher, e.g., 300 dpi, 600 dpi, 1200 dpi, 2400 dpi or higher. This ensures that the generated color data can be displayed in high quality. An appropriate wide color gamut is a color gamut greater than or equal to the standard red-green-blue (sRGB) color gamut. In various embodiments, the screen can be selected to have a color gamut similar to that perceptible to human vision. In one embodiment, the screen of the display device is constructed according to liquid crystal display (LCD) technology, particularly liquid crystal display (LCD) technology further comprising a touchscreen panel. The LCD may be backlit by any appropriate light source. However, the color gamut of the LCD screen can be expanded or improved by selecting a light-emitting diode (LED) backlight or other backlight. In another embodiment, the screen of the display device is constructed according to light-emitting polymer or organic light-emitting diode (OLED) technology. In yet another embodiment, the screen of the display device can be constructed according to reflective display technology such as electronic paper or ink. Known manufacturers of electronic ink / paper displays include E INK and XEROX. Preferably, the screen of the display device also has a reasonably wide field of view so that it can produce an image that does not fade or change significantly when the user views the screen from various angles. Because LCD screens operate by polarization, some models exhibit a high degree of viewing angle dependence. However, various LCD structures have a relatively wide field of view, which may be preferable for that reason. For example, LCD screens constructed according to thin-film transistor (TFT) technology can have a reasonably wide field of view. Also, screens constructed according to electronic paper / ink technology and OLED technology may have a wider field of view than many LCD screens, and may be chosen for this reason.

[0031] In yet another alternative embodiment, the display device is a projection display device configured to project, for example, display, a digital representation of a provided colored coating layer and / or a digital representation of illumination conditions and / or generated color data onto a projection area. In this case, the projection area corresponds to the screen of the display device, as outlined earlier. The projection surface used for projecting the digital representation and / or generated color data may be a 2D or 3D surface such as a flat wall or desk, a car or part thereof, or a defined 3D space. In one example, the generated color data can be projected onto a 3D surface in the form of a car or part thereof. The 3D surface may be colored, translucent, or transparent. In another example, the generated color data is displayed by the projection device using 3D hologram technology, in which a colored 3D object, such as a car or part thereof, obtained by the rendering process described later, is freely projected into a predefined space and can be viewed without the use of 3D glasses or the like. The use of a projection surface corresponding to the shape of the object whose appearance is to be designed, or the use of a colored hologram, can enhance the visual appeal of the generated appearance, as the user can directly compare the appearance of the colored object with the desired appearance of the object. User input on the projection surface can be detected by the projection display device via an image detection system. Suitable projection display devices used for projecting images and / or detecting user interaction with these images are well known in the art.

[0032] In yet another embodiment, as described above, a display device comprising a housing for a screen and a projection display device can be used in combination. In one example, a display device comprising a housing for a screen is used in steps (i) to (iv), and a projection display device can be used in step (v) to display the generated color data. In another example, while the projection display device is used in steps (i) to (iv), the display device comprising a housing for a screen can be used in step (v) to display the generated color data.

[0033] The display device may comprise interaction elements for facilitating user interaction with the display device. In one example, the interaction elements may be physical interaction elements such as an input device or an input / output device, particularly a mouse, keyboard, trackball, touch screen, or a combination thereof. In another example, the interaction elements may be a projection area in which user input in the form of gestures, such as finger gestures or hand movements, is received.

[0034] In one aspect, steps (iv) and (v) are executed simultaneously. "Simultaneously" refers to the time required for the computer processor to execute step (iv) and for the display device to display the generated color data. Preferably, this time is small enough so that within a reasonable time, up to a few seconds, particularly up to 1 second, the user can immediately see the effect of a change in the irradiation conditions on the displayed appearance of the colored object, and thus the irradiation conditions can be interactively changed, and as a result, the desired appearance and / or irradiation conditions can be selected quickly and accurately.

[0035] Step (i): In step (i) of the method of the present invention, the digital representation of the colored coating layer is provided to a computer processor via a communication interface. According to a first embodiment of the method of the present invention, the digital representation of the colored coating layer preferably includes color space data, gloss data, appearance data, texture characteristics, or a combination thereof. An example of color space data is defined by L * a * b * where L * represents lightness, a * represents the red / green appearance, and b * represents the yellow / blue appearance. Another example of color space data is defined by L * C * h, where L * represents lightness, C *'h' represents saturation, and 'h' represents hue. Appearance data may include, for example, long-wavelength values, short-wavelength values, and DOI (Image Clarity). Long-wavelength and short-wavelength values ​​indicate the degree of orange peel, and DOI indicates the degree of gloss and shine. The term "texture characteristics" refers to the roughness and / or gloss characteristics of the effect coating. The roughness and gloss characteristics of the effect coating can be determined, for example, from a texture image acquired by a multi-angle spectrophotometer, as is known in the art. The texture image may be a black and white or a color image.

[0036] According to an alternative embodiment of the method of the present invention, the digital representation of the colored coating layer includes an optimized bidirectional texture function (BTF), which comprises the following steps: - A step of determining the initial BTF of the colored coating layer using a camera-based measuring device; - A step of capturing spectral reflectance data of the colored coating layer for a predetermined number, i.e., a limited number of different measurement shapes, using a spectrophotometer; - To obtain the optimized BTF, the initial BTF in equation (1) is used as follows:

number

number

number

number

number

number

number

number

number

[0037] In the first step, a camera-based measuring device creates multiple images (photographs) of the sample at different field of view, different illumination angles, different illumination colors, and / or different exposure times, thus providing multiple measurement data considering multiple combinations of illumination angle, field of view, illumination color, and / or exposure time. Suitable camera-based measuring devices are commercially available, such as the X-Rite TAC7® measuring device. As the sample, a small flat panel coated with a hardened multilayer coating including a hardened colored coating layer or a colored coating layer coated with a clear coat layer is used. The images obtained from the measuring device are post-processed to obtain an initial BTF. Post-processing may include creating a high dynamic range image from images taken with constant illumination and field of view angles, but with varying illumination color and exposure time. Post-processing may also include correcting the perspective of the photograph relative to the sample and extracting color and texture data from the photograph. Based on the data obtained through post-processing, the parameters of the initial BTF are determined. The bidirectional texture function (BTF) is a six-dimensional function that depends on planar texture coordinates (x,y) and the field of view and illumination spherical angles. Therefore, the BTF is a representation of the texture's appearance as a function of the field of view direction and illumination direction, i.e., the field of view angle and illumination angle. Since the geometric shape of the surface of the object under consideration is unknown and not measured, this is an image-based representation. BTF measurements are typically a collection of images, as they are usually captured by imaging the surface with hemispherical sampling in the possible field of view and illumination directions (see Dana, Kristin J. et al., "Reflectance and Texture of Real-5 World Surfaces," ACM Transactions on Graphics, Vol. 18, 1999, pp. 1-34).

[0038] In the second step, spectral reflectance curves are acquired for a limited number of measurement geometries. Each measurement geometry is defined by a specific illumination angle / direction and a specific field of view angle / direction. Spectral reflectance measurements are performed using, for example, handheld spectrophotometers such as the Byk-Mac I® with six measurement geometries (fixed illumination angles and field of view / measurement angles of -15°, 15°, 25°, 45°, 75°, and 110°), the X-Rite MAT12® with twelve measurement geometries (two illumination angles and six measurement angles), or the X-Rite MA 98® (two illumination angles and up to eleven measurement angles). The spectral reflectance data obtained from these measuring devices is more accurate than the color information obtained from the camera-based measuring device used in the first step.

[0039] In the third step, the initial BTF is segmented (divided) into two main terms (F1) and (F2). The first term (F1) is the homogeneous bidirectional reflectance distribution function.

number

number

number

number

number

[0040] The second term (F2) is a sample

number

[0041] Furthermore, the first term (F1), namely BRDF, further specifies the irradiation and observation directions.

[0042]

number

number

[0043] In the first optimization step, the parameters of the first subterm for each spectral measurement geometry are optimized as follows: - First CIEL from captured spectral reflectance data * a * b * Calculating a value and, - From the initial BTF of equation (1) to the second CIEL * a * b * Calculating a value and, - The first CIELA * b * From the value, the second CIELA * b * By subtracting the values, the correction vectors for the a* coordinate and b* coordinate are calculated, - Interpolate and extrapolate the correction vectors for the entire range of field of view and illumination angle stored in the first sub-term, for each component. - Corrected BTF CIEL * a * b * To obtain the value, the interpolated correction vector is used in the second CIEL of each spectral measurement geometry stored in the first subterm. * a * b * Applying to the value, - Corrected BTF CIEL * a * b * Convert the values ​​to linear sRGB coordinates, and normalize the linear sRGB coordinates. - Storing normalized sRGB coordinates in the first subterm, It may include.

[0044] The multilevel B-spline interpolation algorithm (Lee, Seungyong et al., "Scattered data interpolation with multilevel B30 splines," IEEE Transactions on Visualization and Computer Graphics, Vol. 3, 1997, pp. 228-244) can be used for interpolation and extrapolation of each component of the correction vector.

[0045] In the second optimization step, the parameters of the second sub-term are optimized as follows: - Define a cost function based on the sum of color differences across all spectral reflectance measurement geometries, - First CIEL from captured spectral reflectance data * a * b * Calculating a value and, - From the initial BTF of equation (1) to the second CIEL in different spectral reflectance geometries * a * b * Calculating a value and, - Using a weighted color difference formula, the second CIEL * a * b * Value and the first CIEL * a * b * Comparing with a value, - Optimize the parameters of the second subterm using a nonlinear optimization method so that the cost function is minimized, It may include.

[0046] The cost function C(α,S,F0,a) can be defined over all reflectance measurement geometries according to equation (2):

number

number

number

[0047] As shown in equation (2), the cost function can be complemented by a penalty function designed to account for specific constraints. Such constraints can be used, for example, to keep the parameter values ​​of the intensity function within a valid range and to prevent effects in an optimized BTF that are not observed in reality. In one example, the constraints are defined heuristically. To calculate the color difference, the initial BTF is evaluated on different spectral reflectance geometry, and the resulting CIE L * a * b* The value is obtained from CIE L from spectral reflectance measurements using a weighted color difference formula, such as the formula defined in DIN6157 / 2. * a * b * The parameters of the intensity function are compared with the value and optimized using a nonlinear optimization method, such as the Neldermeer downhill simplex method, to minimize the cost function.

[0048] The first and second optimization steps may be repeated to further improve the accuracy of the optimized BTF. The number of iterations may be specified and predefined. It has been found that reliable good results can be obtained with just three iterations. Since the optimized BTF is more accurate than the initial BTF obtained directly from the camera-based device, when displaying colors using the optimized BTF, higher color accuracy is obtained compared to the measured colors.

[0049] In one aspect of step (i), providing a digital representation of a colored coating layer includes displaying an existing color library on the screen of a display device, selecting a color from the displayed existing library, obtaining a digital representation of a colored coating layer based on the selected color, and providing the obtained digital representation of the colored coating layer to a computer processor via a communication interface. The term “existing color library” refers to a database having a set of pre-selected color values. The existing color library may contain at least two different colors, each corresponding to a color of a colored coating layer prepared from a coating material, each defined by an optimized bidirectional texture function (BTF) calculated as described above, or by color space data and / or gloss data and / or appearance data and / or texture properties.

[0050] Displaying an existing color library may include providing object data for a virtual object, optionally providing further color data, mapping optimized BTFs associated with colors present in the existing library, or mapping color space data and / or gloss data and / or appearance data and / or texture properties associated with colors present in the existing library, optionally mapping further color data to the provided virtual object data, and rendering the mapping results using predefined illumination conditions. Predefined illumination conditions may include direct light sources (also called analytical light sources) such as point light sources, directional light sources or spotlights, or high dynamic range (HDR) environment maps, particularly high dynamic range (HDR) environment maps. Rendering processes using direct light sources are known in the art and can be performed in real time (see, for example, the OpenGL shading language, Rost, RJ et al., AddisonWesley Professional, 2009). Rendering processes that use high dynamic range (HDR) environment maps are known in the technical realm as image-based lighting (see, for example, Debevec, Paul, "Image-based lighting," IEEE Computer Graphics and Applications, March / April 2002, pp. 37-34). In these processes, real-world illumination is captured as an omnidirectional, high-dynamic-range image, and this illumination is mapped to a representation of the environment. Computer graphics objects are then placed within the environment, and the light from the environment illuminating the computer graphics objects is simulated, so that real or virtual objects are illuminated with an image of light from the real world. Therefore, rendering results using image-based lighting are perceived as more realistic because they use real-world illumination.Therefore, to obtain a more realistic image of a colored object, it may be preferable to perform the rendering process using image-based lighting. The virtual object can be selected from virtual 2D objects such as colored regions, or virtual 3D objects such as color chips, dome shapes, car bodies, or parts of such car bodies. The car body or part thereof may be a general car body or part thereof, or a specific car body or part thereof. The term "general car body or part thereof" refers to a car body or part thereof that is a general representation of a vehicle class, such as a car or motorcycle. Such general-purpose bodies are used only to represent the general shape of each vehicle class and are not typically manufactured. In contrast, the term "specific car body or part thereof" refers to a car body or part thereof that has the same shape as an actual manufactured car body. The data for the virtual object may be stored in the memory of the display device or in a computer-readable medium such as a database connected to the display device via a communication interface. In one example, the user can select a virtual object before rendering. For this purpose, available virtual objects are displayed to the user on the screen of the display device, and the user can select the desired object through interaction elements. Object data associated with the selected virtual object is then retrieved by the processor from a computer-readable medium such as a database or internal memory before rendering. In another example, a predefined virtual object is used for rendering. In yet another example, the user can provide the computer processor with a virtual object stored in a data storage medium via a communication interface. The virtual object used for rendering may be a colored virtual object. In one example, color data associated with an available or predefined virtual object is retrieved by the processor based on the selected or predefined virtual object. In yet another example, the user can select a color from an existing color library displayed on the screen of a display device, as disclosed earlier.Next, color data associated with the selected color is retrieved from a computer-readable medium such as the display device's memory or a database before rendering. By using a colored virtual object, if the selected color is further combined with a colored coating layer, the user can see the overall visual impression of the object. If the colored coating layer is a colored clear coat and is combined with a colored base coat layer underneath, this may be preferable because the color of the base coat layer is at least partially visible through the colored clear coat layer due to its at least partial transparency. The displayed existing color library can be manipulated, for example, by zooming, moving, rotating, and / or scrolling, to enhance user comfort when browsing the displayed colors present in the existing library.

[0051] An existing color library can be stored in a computer-readable medium and provided via a communication interface to a computer processor that performs the rendering steps disclosed earlier. The computer-readable medium may be the memory of the display device, or an external storage device such as a database connected to the display device via a communication interface, particularly a wireless communication interface. The user can select a color from the displayed existing color library via interaction elements that reside within the display device and are coupled to the display device via a communication interface, or are projected by the display device. After selecting a desired color from the displayed existing color library, a digital representation of the colored coating layer can be obtained by searching the database for the digital representation based on the selected color. By displaying a predefined color library, the user can easily browse all available colors and select a desired color in an intuitive manner.

[0052] In an alternative embodiment of step (i), the step of providing a digital representation of a colored coating layer includes providing coating layer identification data, obtaining a digital representation of the coating layer based on the provided coating layer identification data, and providing the obtained digital representation. Providing the coating layer identification data may include providing an optimized bidirectional texture function (BTF) of the colored coating layer, and / or providing data indicating the colored coating layer. The data indicating the colored coating layer may refer to a color number, color name, QR code (registered trademark), barcode, etc. Such data may be provided by the user via the screen of a display device. The screen may have a GUI to facilitate user data input. The digital representation of the coating layer can then be obtained by searching a database of the digital representations based on the provided coating layer identification data.

[0053] In one embodiment, step (i) further includes displaying a color associated with a digital representation of a provided colored coating layer on the screen of a display device. "Color associated with a provided digital representation" means a display color obtained by using an optimized BTF contained in the digital representation of the colored coating layer to display a color on the screen of a display device. Displaying the obtained color data may include providing object data for a virtual object and rendering the provided color data and the provided object data using predefined illumination conditions. The predefined illumination conditions, rendering process, and virtual object may be the same as those described above. In this embodiment, the user can select a virtual object that corresponds to a real object whose appearance should be designed so that the visual impression of the colored virtual object is displayed as realistically as possible. Furthermore, this allows the user to verify whether the selected color corresponds to a desired color, and therefore, if an incorrect starting color is selected, for example by entering incorrect color identification data or by selecting the wrong color from the existing library described above, the user can correct the starting color.

[0054] Step (ii): In step (ii) of the method of the present invention, a digital representation of irradiation conditions is provided by displaying a user interface including multiple irradiation conditions on a display device, detecting user input indicating that an irradiation condition has been selected from the displayed multiple irradiation conditions using a computer processor, and in response to the detected user input, obtaining a digital representation of the irradiation condition associated with the detected user input via a communication interface using the computer processor. The term "multiple irradiation conditions" means at least two different irradiation conditions, i.e., at least two irradiation conditions that are different from each other. The user interface may be displayed on the display device based on a user interface generation generated by the processor. The user interface generation may include additional buttons, menus, etc., separate from the multiple irradiation conditions. The multiple irradiation conditions may be depicted within the presentation of the user interface using icons, images, text, or a combination thereof.

[0055] In one embodiment, the illumination conditions include at least (i) ambient illumination surrounding the display device, (ii) a predefined high dynamic range (HDR) environment map, and (iii) ambient light associated with a specific time, date, and location.

[0056] In one embodiment, user input is detected via an interaction element. The interaction element may be a physical input device, an input / output device, or a projected input device, as described above.

[0057] In one embodiment, obtaining a digital representation of the irradiation conditions is possible. - To obtain data via a communication interface indicating the date, time, location, in particular geographic location, and optionally the degree of haze, and / or - To acquire data acquired from at least one illumination sensor of the display device, data acquired from at least one orientation sensor of the display device, and optionally data acquired from at least one viewpoint sensor via a communication interface, and / or - To obtain at least one high dynamic range (HDR) environment map via a communication interface. Includes.

[0058] The illumination sensor and viewpoint sensor of the display device may be implemented as a CMOS imaging module, an embedded camera, or a similar device as described above, respectively. The orientation sensor may be located within the housing of the display device and may include any suitable type of sensor capable of sensing the movement of the display device, such as pitch, roll, and yaw around one or more axes of the display device. Such sensors may include micro-electromechanical (MEM) gyro sensors, such as those available from INVENSENCE CORP. in Santa Clara, California, in place of or in addition to gyro sensors, as well as inclinometers, accelerometers, etc. A high dynamic range (HDR) environment can be obtained by using a special camera equipped with a rotating objective lens or a fisheye lens to capture a sphere that reflects the scene, or by stitching images together to transform the image into a sphere that surrounds the scene.

[0059] In one example, the step of acquiring location, particularly geographical location data, via a communication interface may include displaying a world map on a display device, detecting user input via an interaction element indicating the selection of a location on the displayed world map, and, in response to the detected user input, acquiring data associated with the selected geographical location, particularly GPS data, via a communication interface with a processor. The world map may be displayed in response to user input indicating the selection of an icon displayed on the display device (corresponding to one of several illumination conditions displayed on the user interface). The world map may be stored in the display device's internal memory or in a data storage medium such as an external database, or may be provided via a web access service such as Google Maps. To facilitate finding a desired location on the displayed world map, the user can use interaction elements to zoom or move the world map, or enter a search string such as a city name. In another example, the location of the display device can be determined using a location determination module such as GPS or a cellular module, and the determined location can then be acquired via a communication interface with a computer processor.

[0060] Acquiring data indicating the date and / or time and / or the degree of empty haze via a communication interface may include displaying at least one adjustment tool containing at least one regulator corresponding to the date and / or time or haze, detecting user input indicating operation of at least one adjustment tool via an interaction element, in particular by detecting user input by detecting the movement of at least one regulator of the displayed adjustment tool via an interaction element, and determining the date and / or time or haze associated with the position of each regulator in response to the detected user input. "Adjustment tool" may refer to a part of a graphical user interface that enables the modification of the displayed date and / or time or empty haze. The use of the adjustment tool provides user guidance regarding the operation of the date and / or time and / or haze, and thus presents a user-intuitive way of setting the desired parameters. The step of operating at least one adjustment tool via an interaction element may include adjusting the date and / or time and / or haze via the interaction element, for example, by moving each regulator of the adjustment tool using the interaction element. In one example, the user can freely select the time and / or date and / or haze by operating an adjustment tool. In another example, at least one of the date, time, or haze may be fixed, i.e., a predefined value such as the actual time or actual date or a predefined haze related to the selected location / time may be obtained from a data storage medium via a communication interface with a computer processor.

[0061] Data acquired from at least one illumination sensor of the display device may include data about the illumination conditions surrounding the display device, such as lux level, spectral components, illumination direction, and at least one photograph of the environment surrounding the display device, particularly at least one high dynamic range (HDR) or low dynamic range (LDR) photograph, or a combination thereof. The lux level may represent the general level of ambient light incident on the surface of the display device's screen. Spectral content may represent the spectral components of the ambient light. Illumination direction may include the primary illumination direction, i.e., the main direction from which ambient light is incident. If the appearance of an object is modeled in an environment with one or a few ambient light sources, there may be one or a few clear primary illumination directions. On the other hand, if the appearance of a surface is modeled in a diffuse ambient environment with multiple or diffuse light sources, there may be no primary illumination direction. By using data acquired from illumination sensors, it becomes possible to consider the influence of actual illumination sources present in the environment surrounding the display device when predicting the appearance of a colored object, instead of using predefined illumination sources.

[0062] Data acquired from at least one viewpoint sensor of the display device may include at least one photograph showing a user viewing the display device. According to various embodiments, the illumination sensor and viewpoint sensor may be implemented as a single sensor having both favorable point position information and illumination information derived from at least one photograph acquired by a single sensor.

[0063] Acquiring at least one high dynamic range (HDR) environment map may include displaying at least one high dynamic range (HDR) environment map on the screen of a display device, detecting user input indicating that the displayed high dynamic range (HDR) environment map has been selected via an interaction element, and, in response to the detected user input, acquiring the high dynamic range (HDR) environment map associated with the detected user input via a communication interface. The data associated with the selected HDR environment map may include existing HDR environment maps, i.e., HDR environment maps previously acquired and stored in a data storage medium. The step of displaying at least one HDR environment map on a display device may include displaying at least one existing HDR environment map on the display device. Existing HDR environment maps may be stored in a data storage medium such as the internal memory of the display device, or may be acquired from a database via a communication interface before display. Acquiring a selected HDR environment map may include acquiring the selected HDR environment map from a data storage memory such as an external database or internal memory based on user input detected via a communication interface with a computer processor.

[0064] In one embodiment, step (ii) may further include displaying the acquired illumination conditions on a display device. In one example, this may include calculating an HDR environmental map from data indicating the date, time, location, in particular geographic location, and optionally the degree of haze in the sky, using a model derived from past illumination conditions as described above, and displaying the calculated HDR environmental map. In another example, this may include calculating an HDR environmental map from data acquired by illumination sensors and / or orientation sensors and / or viewpoint sensors, using at least one model derived from past environmental conditions as described above, and displaying the calculated HDR environmental map. In yet another example, this may include displaying an acquired existing HDR environmental map. By displaying the illumination conditions provided to the user on a display device, the user can verify whether the desired illumination conditions have been selected and modify the input as necessary.

[0065] Any step (iii): In an optional step (iii) of the method of the present invention, models derived from past irradiation conditions and / or models derived from past environmental conditions are provided to a computer processor via a communication interface. This step may also be performed after providing a digital representation of the irradiation conditions to the computer processor, as described in relation to step (iii) below. The communication interface may be wired or wireless, and may be particularly wireless. Examples of wireless communication interfaces include WLAN, WiFi, or Bluetooth®. The computer processor may be any suitable type of processor. According to various embodiments, the computer processor, in particular the processor of a display device, may include a graphics processing unit (GPU) specifically designed to handle graphics processing. For example, suitable GPUs are available from NVIDIA and AMD (Advanced Micro Devices, Inc.). The processor may also communicate with interaction elements such as memory and input / output devices. Input / output devices may allow a user to configure the device and / or input data. In various embodiments, the display device may provide a menu-driven user interface on the screen or on a secondary display that allows a user to input information and guide the user through the process. In addition to other peripherals, the processor can communicate with the computer via wired or wireless data links, such as RS232 or Universal Serial Bus (USB) links.

[0066] In one aspect of step (iii), the model derived from past illumination conditions is a physically based analytical model of the daytime sky. Such models are known in the art and are commonly used in rendering processes that involve the use of daylight. One suitable model is the Preetham model of equation (3),

number

[0067] Parameters A through E are analytically calculated from a linear function that takes turbidity as an argument and returns parameters A through E, and from a quadratic function of turbidity and solar elevation angle for calculating zenith radiance. In addition to radiance, the Preetham model also provides two chroma channels calculated using the same approach, and the output can be converted to spectral radiance data. Another suitable model is the one described in I. Hosek et al., "An analytic model for full spectral sky-dome radiance," ACM Transactions on Graphics, 2012, Vol. 31, Article No.: 95, https: / / doi.org / 10.1145 / 2185520.2185591. The latter model yields good results for both high and low atmospheric turbidity values, and therefore, it is preferable to use the latter model in this invention. Hosek et al.'s model provides spectral radiance data for each point and wavelength on a hemisphere, which can be used to compute an HDR environment map that arbitrarily fills the lower half of a spherical panorama with a predefined bottom color. For this purpose, spectral radiance data for the pixels of the HDRI is computed, and the computed spectral radiance data for each pixel is either an sRGB value or a CIEL value. * a * b * It is converted into color space data such as values.

[0068] In one aspect of step (iii), at least one model derived from past environmental conditions provides a relationship between data acquired from the illumination sensor and / or orientation sensor and / or viewpoint sensor of the display device and the ambient light conditions surrounding the display device. "Ambient light" may refer to any light source not explicitly supplied to the environment for the purpose of determining the ambient light of the display device using the aforementioned model. This term typically refers to light sources that are already naturally available (e.g., sun, moon, lightning) or artificial light already in use (e.g., for illuminating a room). The ambient light conditions surrounding the display device can be determined from data acquired by the illumination sensor and / or viewpoint sensor, such as a CMOS imaging module, embedded camera, or similar device of the display device, using a number of models commonly known in the art. In one example, the illumination sensor and viewpoint sensor may be implemented as a single sensor. The data acquired by the sensor may include high dynamic range (HDR) images or low dynamic range (LDR) images. One suitable model is, for example, P. Debevec, "Image-based lighting," ACM SIGGRAPH 2006 Courses, pages 4-es, and Q. Yang et al., "Inertial sensors aided image alignment and stitching for panorama on mobile phones," Proceedings of the 1st International Workshop on Mobile Location-Based Service, 2011, pp. 21-30 (hereinafter referred to as the HDRI model). Another suitable model derives color and brightness from a low dynamic range (LDR) photograph and uses these values ​​to estimate the ambient light conditions surrounding the display device (hereinafter referred to as the ambient light model). An example of such an ambient light model is implemented in Apple's ARKit program library.Another suitable model calculates spherical harmonic (SH) coefficients from low dynamic range (LDR) photographs (hereinafter referred to as the SH coefficient model). Considering that irradiance changes smoothly with orientation, only nine coefficients are needed to accurately represent the surface irradiance, making this a very effective method for compressing direct illumination from distant light sources. In one example, the SH coefficients can be calculated from the color, brightness, and primary illumination direction present in the data acquired by the illumination and / or viewpoint sensor, as implemented in Apple's ARKit program library, where the primary illumination direction is determined using the shadow of a face recognized in acquired data such as a photograph. The calculated SH coefficients can be used to generate color data in step (v) (see, e.g., OpenGL Shading Language, Rost, RJ, et al., AddisonWesley Professional, 2009), or the HDR environment map can be calculated from the SH coefficients before generating color data in step (v) (see, e.g., P.-P. Sloan, "Stupid Spherical Harmonics (SH) tricks", Game Developers Conference 2008, February 2008). In another example, SH coefficients and HDR environment maps can be calculated from low dynamic range (LDR) photographs in a limited field of view (FOV) using a trained neural network, such as the one implemented in Google's ARCore program library (see, for example, C. LeGendre et al., "DeepLight: Learning Illumination for Unconstrained Mobile Mixed Reality," Proceedings of the IEEE / CVF Conference on Computer Vision and Pattern Recognition, 5918-5928).

[0069] Step (iv): In step (iv) of the method of the present invention, color data for a colored coating layer is generated by a computer processor based on a digital representation of a provided colored coating layer, a digital representation of provided illumination conditions, and an optionally provided model. In one embodiment, the step of generating color data by a computer processor includes providing object data for a virtual object, providing additional color data for optionally at least one further coating layer, mapping the digital representation of the provided colored coating layer and the optionally additional color data onto the provided virtual object, and rendering the mapping result using the digital representation of the provided illumination conditions and an optionally provided model.

[0070] The virtual object may be a 2D virtual object such as a geometric shape, or a 3D virtual object such as a chip, dome shape, car body, or part thereof. The virtual object may be a predefined virtual object, which may be selected by the user from predefined objects displayed before mapping, or provided by the user before mapping as described above. The step of providing further color data may include selecting at least one further color for the colored coating layer, in particular from an existing color library as described above, and providing the color data associated with the selected color to the processor before rendering. This allows the user to model the appearance of a colored object containing multiple colored coating layers.

[0071] Depending on the digital representation of the provided illumination conditions, the data contained in the digital representation is either processed by a computer processor using an appropriate provided model before rendering, or used directly for rendering. In one example, the data contained in the digital representation of the provided illumination conditions may be processed using the display device's processor. This may be preferable if the display device's processor has sufficient computing power to perform pre-rendering data processing within a reasonable time, such as a few milliseconds to a few seconds. In another example, the processing is performed by a further processor located outside the display device. This may be preferable if the computing power of the display device's processor is insufficient to perform pre-rendering data processing within an acceptable time.

[0072] In one embodiment, step (iv) further includes, before generating color data for the colored coating layer, calculating the ambient illumination conditions surrounding the display device using a computer processor from a digital representation of the provided illumination conditions and at least one provided model derived from past environmental conditions. In this case, the digital representation of the provided illumination conditions includes data acquired by at least one sensor, which is an illumination sensor, an orientation sensor, and a viewpoint sensor. The ambient illumination conditions are calculated by a computer processor from the acquired sensor data using at least one model derived from past environmental conditions as described above, such as an ambient light model, an HDR environment map model, or an SH coefficient model. In one example, calculating the ambient illumination conditions may include calculating the ambient light by estimating brightness and color from data acquired by the illumination sensor as described above. In another example, calculating the ambient illumination conditions may include calculating an HDR environment map from data acquired by the illumination sensor and orientation sensor as described above. In yet another example, calculating ambient light conditions may include calculating spherical harmonic (SH) coefficients from data acquired by an illumination sensor and optionally from data acquired by a viewpoint sensor, and optionally, calculating an HDR environment map from the calculated SH coefficients as described above.

[0073] The viewpoint position can be derived from a photograph acquired by a viewpoint sensor, or by a combination of illumination from the display device and a viewpoint sensor. For example, a human eye can be identified in a photograph according to any appropriate algorithm. It may be assumed that the eye is directed towards the screen of the display device. The distance between the eye and each point on the surface of the display device's screen can be assumed based, for example, on the orientation of the display device as sensed by at least one orientation sensor. For example, when a user holds the display device at a position close to eye level, they tend to initially hold it at a more upright angle than when holding it near their waist. From the position of the eye, the orientation of the eye, and the distance between the eye and each point on the surface of the display device in at least one photograph, the viewpoint position can be derived.

[0074] Rendering may be performed using image-based lighting (IBL), as described above, with a provided or calculated HDR environment map (see, e.g., Debevec, Paul, "Image-based lighting," IEEE Computer Graphics and Applications, March / April 2002, pp. 37-34), or with a calculated SH coefficient (see, e.g., OpenGL Shading Language, Rost, RJ, et al., AddisonWesley Professional, 2009). Rendered objects may be placed in front of a predefined background, or a background selected by the user via a GUI displayed on the screen of a display device. The background may be a uniform color containing different colors, or a photograph of the environment. User background selection may include selecting a background from a set of predefined backgrounds displayed via an interaction element, or providing a background by the user. The selected background may then be retrieved from a data storage medium based on the user's selection and provided to the computer processor. User-provided backgrounds may include acquiring backgrounds such as images stored from data storage media, or acquiring photographs of the environment using the illumination sensor of a display device, and providing the acquired background or photographs to a computer processor. By selecting a background, the user can display rendered objects in a desired environment and thus obtain a better impression of the object's appearance under real-world conditions.

[0075] In one example, rendering is performed by the display device's processor. If the ambient illumination conditions are calculated by this processor as described above, the processor can use the calculated ambient illumination conditions directly for rendering. If the ambient illumination conditions are calculated by a further processor, the calculated ambient illumination conditions are provided to the display device's processor via a communication interface before rendering. In another example, rendering is performed by a further processor located outside the display device, and the rendering results are provided to the display device's processor via a communication interface for display on the screen. This may be preferable if the display device's processor's computing power is insufficient to perform rendering within an acceptable time, i.e., if the display device's processor cannot render at least 25 images within one second.

[0076] In one aspect of step (iv), the generated color data is recalculated in response to changes in data included in the digital representation of the illumination conditions provided in step (iii), particularly in response to changes in data provided by at least one illumination sensor, and / or in response to changes in data provided by at least one orientation sensor, and / or in response to changes in data provided by at least one viewpoint sensor, and displayed on the screen of the display device. Recalculation may be performed automatically, i.e., without user interaction, when it is detected that the data included in the digital representation of the provided illumination conditions, particularly the data acquired by at least one sensor, has been changed, for example, by changing the illumination conditions surrounding the display device, and / or changing the orientation of the display device, and / or changing the viewpoint. For this purpose, the computer processor may be programmed to detect changes in data acquired by at least one sensor, or changes in user-inputted data such as time, position, location, haze, etc., and may initiate recalculation of the generated color data using the newly acquired sensor data or the modified input. Recalculation and display of the recalculated color data can be performed in real time or near real time. In this way, when the user tilts or rotates the display device, or changes the viewpoint or time / position / haze, the appearance of the colored object displayed on the display device's screen may behave as if the user were tilting or rotating an actual sample, changing the viewpoint on an actual sample, or changing the time / position / haze of an actual sample.

[0077] Step (v): In step (v), the color data generated in step (iv) is displayed on the screen of the display device. In one embodiment, the step of displaying the color data received from the computer processor on the screen of the display device includes mapping the generated color data, in particular each rendering point, to the screen of the display device and displaying the rendered object on the screen of the display device. If the object is two-dimensional, such as a plane, there may be a one-to-one correlation between each rendering point and a pixel or group of pixels on the screen of the display device. However, in various embodiments, the number of rendered points may exceed the number of available pixels. In this case, the appearance of multiple points can be averaged, or otherwise aggregated into a single pixel or group of pixels. In three-dimensional objects, due to the shape of the object, multiple rendering points may be mapped to the same pixel or group of pixels. When this happens, appropriate corrections can be made according to any appropriate method. For example, if one point on a surface tends to obscure another point relative to the viewpoint, the relevant pixel or group of pixels can display the appearance of the point closest to the viewpoint. Alternatively, for example, pixels mapped to multiple points can display the average or other aggregation of the point's appearance. Furthermore, when a three-dimensional object is mapped onto the screen of a display device, the mapping may be based on facets, vertices, or any other suitable method of representing the three-dimensional object.

[0078] If the color associated with the digital representation is already displayed on the display device screen in step (i), or if the aforementioned recalculation has been performed, the step of displaying the color data received from the processor on the display device screen may include automatically updating the color displayed on the display device screen in step (i) in response to performing step (v) or in response to the recalculation performed in step (iv). Automatic updating may mean updating the displayed color without any user interaction required for the update. Automatically updating the color displayed on the display device screen may include mapping the generated color data generated in step (iv), in particular each rendering point, to the display device screen, so that the color displayed in step (i) is updated with the rendering result of step (iv), or the color displayed in step (v) is updated with the rendering result of the recalculation performed in step (iv).

[0079] In one embodiment, step (v) further includes storing the generated color data and / or further data in a computer-readable medium. The further data may include ambient light conditions calculated from a digital representation of the provided illumination conditions using one of the aforementioned models, as well as data used in the calculation, such as sensor data or user-inputted data. Storing the generated color data and / or further data can speed up the prediction process because the stored data can be quickly retrieved when needed again and does not need to be generated by calculation. The data may be associated with the digital representation of the colored coating layer and the digital representation of the illumination conditions and stored in a database accessible to the computer processor before the color data is generated. For example, the computer processor can access the database before generating the color data as described above and check whether the generated color data and / or calculated ambient light conditions are available based on the digital representation of the provided colored coating layer and the digital representation of the provided illumination conditions. This allows for the construction of a database containing the generated color data and / or calculated ambient light conditions during use of the method of the present invention, and speed up step (iv) can be improved because the color data and / or ambient light conditions do not need to be generated by the computer processor but can instead be quickly retrieved from the database.

[0080] In one embodiment, step (v) further includes generating a modified color library by adding the displayed colors to an existing color library or a newly generated color library. The newly generated color library may be generated by the user before the first colors are added to the library and therefore does not include any colors prior to the addition of the colors displayed in the library. The modified color library can be associated with a saved user profile for future use when designing the colors of a colored coating layer. This allows the user to save and retrieve predicted colors and modify predicted colors at a later point in time. The user can also remove at least one added color from the modified color library. In one example, the user can also create a sorted color library by sorting the colors present in the existing color library or the modified color library according to a grouping criterion. The sorted color library can be associated with a saved user profile for future provision of the sorted color library. The sorted color library can represent a favorites list. The grouping criterion may be arbitrarily selected by the user or may be a predefined criterion displayed on the screen for user selection, such as newly designed colors, recently selected colors, etc.

[0081] Further steps In one embodiment, steps (i) through (v), or steps (ii) through (v), or steps (i), (iv) and (v) are repeated. This allows for the selection of a new color and different illumination conditions (by repeating steps (i) through (v)), the modification of the illumination conditions for the selected color (by repeating steps (ii) through (v)), and the modification of the color of the selected illumination conditions (by repeating steps (i), (iv) and (v)). When steps (i) through (v), steps (ii) through (v), or steps (i), (iv) and (v) are repeated, the step of displaying the generated color data on the screen of the display device may include automatically updating the color displayed on the screen of the display device in step (v) in response to the repetition of steps (i) through (v), or the repetition of steps (ii) through (v), or the repetition of steps (i), (iv) and (v). By automatically updating the color displayed in step (i) or (v), the effect of illumination conditions on a given color can be visualized, and thus interactive guidance can be provided to the user when selecting a desired color.

[0082] In one embodiment, the method of the present invention further involves the following steps: (vi) A step of modifying the provided digital representation of the colored coating layer in order to generate a modified digital representation of the colored coating layer; (vii) A step which optionally repeats step (vii); (viii) A step of generating color data for the coating layer based on a modified digital representation of the colored coating layer; (ix) The step of displaying the generated color data received from the processor on the screen of a display device; (x) A step which optionally repeats steps (vii) through (ix); (xi) Optionally, a step of determining with a computer processor whether the corrected digital representation of the colored coating layer is within at least one predefined tolerance; (xii) Optionally, according to the determination that the modified digital representation is within at least one predefined tolerance: the steps (ii) through (v) are repeated using the modified digital representation of the colored coating layer; (xiii) Optionally, in accordance with the determination that the modified digital representation is within at least one predefined tolerance: the step of providing the modified digital representation to a coating material manufacturing site via a communication interface and optionally manufacturing a coating material based on the provided modified digital representation; (xiv) Optionally, in accordance with the determination that the modified digital representation is outside of at least one predefined tolerance: the step of displaying at least one recommendation on the screen of the display device; (xv) Optionally, according to the determination that the modified digital representation is outside at least one predefined tolerance: a step repeating step (vi) to (ix), or step (vi) to (x), or step (vi) to (xiv), Includes.

[0083] Step (vi): Step (vi) of modifying the provided digital representation of the colored coating layer in order to generate a digital representation of the colored coating layer is as follows: - Optionally, to acquire data regarding the formulation of colored coating materials related to the provided digital representation and to provide the acquired data to a computer processor via a communication interface; - To display at least some of the components contained in the coating material used to prepare the colored coating layer on the screen of a display device; - Manipulate at least one displayed component via an interaction element; - The computer processor detects the operation; - The computer processor converts the detected operations into a modified digital representation of the colored coating layer, It may include.

[0084] Data relating to the formulation of the coating material used to prepare the colored coating layer may include data relating to the types and amounts of at least some of the components present in the coating material. The colored coating material typically contains at least one pigment, at least one binder, and at least one solvent. Further components may include, for example, fillers, matting agents, crosslinking agents, and additives. If the provided digital representation of the colored coating layer does not contain such data, this data may be obtained from a database associated with each digital representation of the colored coating layer provided in step (ii).

[0085] Displaying at least some of the components present in a coating formulation used to prepare a colored coating layer on the screen of a display device may include displaying at least one adjustment tool including multiple regulators, each regulator corresponding to the type and amount of components present in the coating material. The at least one adjustment tool may be generated from a digital representation of a provided coating layer. This may include determining data on a computer processor regarding the formulation of the coating material used to prepare the colored coating layer, which is contained in the digital representation of the provided colored coating layer, and generating at least one adjustment tool based on the determined data. In one example, the adjustment tool may be a box having multiple visually distinct compartments, each compartment indicating a type of component present in the coating material, the size of each compartment indicating the amount of each component, and the multiple regulators corresponding to visual elements, particularly lines, that separate the compartments. The types of components may be displayed within the compartments using graphic representations indicating the types of components. The graphic representations may be obtained from a digital representation of a provided colored coating layer and may be selected from images such as color chips of colored pigments, metal pieces of metallic effect pigments, glass pieces of glass flakes, or solid blocks of binders. By using boxes with sections of different sizes and containing graphic representations of specific components, an overview of the relevant components becomes easier, and by changing the size of the sections and / or the components present within them, the influence of each component type / amount on the resulting color of the coating layer can be easily evaluated. Using the formulation of the coating material as a basis for modification allows for the investigation of the influence of color on coloring, and as a result, the desired visual appearance can be designed in a very intuitive way.

[0086] Manipulating at least one displayed component via an interaction element may include adjusting the type and / or quantity of at least one displayed component via an interaction element. For example, this may include moving at least one regulator of at least one displayed adjustment tool via an interaction element.

[0087] In one example, an operation may be detected by a processor connected to an interaction element via a communication interface and provided to a computer processor via a further communication interface connecting the two processors. This is preferable when a touch panel is used, which includes a panel processor that detects touchscreen gestures. The detected touchscreen gestures are provided via the communication interface to a computer processor that may be located inside or outside the display device. In another example, user input may be detected by a computer processor located inside the display device. This is preferable when an external input device is used as an interaction element.

[0088] The detected user input is converted by a computer processor into a modified digital representation of the colored coating layer. This may include converting the detected user input into modified formulation data of the colored coating material used to prepare the colored coating layer. The modified formulation data of the colored coating material preferably includes numerical values ​​and reflects the modification of the coating material components performed by the user as described above. Thus, converting the detected user input may include modifying the numerical values ​​of the formulation data of the unmodified colored coating material, where the modification reflects the detected user input. In one example, the conversion is performed by a computer processor located inside the display device. In another example, the conversion is performed by a further computer processor located outside the display device, particularly inside another computing device, by providing the user input to the further processor via a communication interface and performing the conversion based on the provided user input.

[0089] Step (vii): In any step (vii), step (vi) is repeated at least once. This is preferable if at least one additional colored coating layer is present on the object and the color of at least one additional coating layer should also be modified.

[0090] Step (viii): In step (viii), the color data for the colored coating layer is generated by a computer processor based on the modified digital representation of the colored coating layer. If step (vii) is repeated, the user can choose whether to generate the color data for the coating layer in step (viii) based on all modified digital representations obtained by performing step (vii) at least twice, or to generate the color data for the selected modified digital representation in step (viii). In the former case, the generated color data displayed in step (ix), described below, corresponds to the visual impression of the combination of colored coating layers. In the latter case, the generated color data displayed in step (ix) corresponds to the visual impression of the selected colored coating layer, and the color data for the modified digital representations not selected in step (viii) may be generated by repeating (ix) and (x). For example, generating color data based on the modified digital representation of the colored coating layer may include obtaining color data based on the modified digital representation, in particular, by obtaining color data from a database or lookup table based on the modified formulation data of the provided colored coating material, thereby obtaining color data based on the modified formulation data of the colored coating material. This may involve comparing the acquired color data with a predefined tolerance, such as color distance, appearance distance, or a combination thereof. In another example, generating color data based on a modified digital representation of a colored coating layer may involve using a data-driven model parameterized on color data of past coating layers and past formulations of the colored coating material used to prepare those past coating layers, in order to compute color data from the modified digital representation of the colored coating layer, particularly from the modified formulation data of the colored coating material. A “data-driven model” may refer to a model that is at least partially derived from data. Using a data-driven model allows for the description of relationships that cannot be modeled by physicochemical laws. Using a data-driven model allows for the description of relationships without solving equations from physicochemical laws. This can reduce computational power and improve speed.Data-driven models may be derived from statistics (Statistics 4th edition, David Freedman et al., WW Norton & Company Inc., 2004). Data-driven models may be derived from machine learning (Machine Learning and Deep Learning frameworks and libraries for large-scale data mining: a survey, Artificial Intelligence Review, Vol. 52, 2019, pp. 77-124). Data-driven models may include empirical models or so-called "black-box models." An empirical model or "black-box" model may refer to a model constructed using one or more of machine learning, deep learning, neural networks, or other forms of artificial intelligence. An empirical model or "black-box" model may be any model that yields a good fit between training data and test data. Alternatively, a data-driven model may include an exact model or a "white-box" model. An exact model or "white-box" model refers to a model based on physical and chemical laws. Physical chemical laws may be derived from first principles. Physical chemical laws may include one or more of the following: chemical reaction kinetics, the law of conservation of mass, the law of conservation of momentum and energy, particle ensembles in any dimension, and physical and / or chemical relations. A rigorous model or a “white-box” model may be selected according to the physical chemical laws governing the respective problem. Data-driven models may include hybrid models. A “hybrid model” refers to a model that includes both white-box and black-box models; see, for example, the review article by Von Stoch et al., Computers & Chemical Engineering, Vol. 60, 2014, pp. 86-101.Parameterized data-driven models of past coating layer color data and past formulations of colored coating materials used to prepare past coating layers are well known in the art and are disclosed, for example, in US20020184167A1 and US20090078936A1.

[0091] Step (ix): In step (ix), the generated color data is displayed on the screen of the display device. This may include providing object data for a virtual object, optionally providing additional color data for at least one further coating layer, and rendering the generated color data, the provided object data, and optionally the further provided color data using the predefined illumination conditions as described above. Rendering may be performed by the processor of the display device or a further processor, as described above. This step may include automatically updating the colors displayed in step (v) in response to performing steps (vi) to (ix) as described above. Step (ix) may include saving the generated color data and / or the displayed color data, adding the displayed color data to an existing color library or a newly generated color library, and further steps related to the modified color library, as described above in relation to step (v).

[0092] Any step (x): In any step (x), steps (vii) through (ix) are repeated. This allows the formulation of the coating material to be modified until the desired color is obtained.

[0093] Any step (xi): In an optional step (xi), the computer processor determines whether the modified digital representation of the colored coating layer is within a predefined tolerance. This ensures that the color designed by the user meets a certain predefined tolerance. Such tolerances can be selected from the maximum or minimum amount of an ingredient, the acceptable combination of at least some of the ingredients, and combinations thereof. The ingredients may be pigments, binders, solvents, or further ingredients present in the coating material used to prepare the colored coating layer. In particular, the maximum or minimum amount of a particular pigment type and / or binder, and / or the acceptable combination of pigment types and / or binders, can be selected as predefined tolerances. The predefined tolerances may be stored in a storage device such as a database and provided to the computer processor performing step (xi) via a communication interface. The step of determining whether the modified digital representation of the coating layer is within at least one predefined tolerance may include comparing data of the modified formulation of the coating material contained in the modified digital representation with at least one predefined tolerance using a computer processor, particularly a computer processor of a display device. The comparison may be performed using a strict model. A rigorous model can be selected according to the physical and chemical laws governing each particular problem.

[0094] Any step (xii): In any step (xii), steps (ii) through (v) are repeated if the modified digital representation is within at least one, in particular all, predefined tolerances. This allows the user to verify whether the modified color has the desired appearance even under defined illumination conditions. In one embodiment, any step (xii) may also be performed before step (xi). However, it is desirable to perform step (xii) after step (xi) because it is ensured that the color designed by the user meets the defined criteria and can be manufactured and applied to an object, and therefore, if desired by the user, a colored object having the designed color can be manufactured.

[0095] Any step (xiii): In an optional step (xiii), a modified digital representation of the colored coating material is provided to the coating material manufacturing site via a communication interface. In one example, this may be triggered by the user when the user is satisfied with the appearance of the colored object displayed on the screen of the display device (i.e., when a modified digital representation of the colored coating layer was provided in step (ii)) after repeating steps (ii) to (v) using the modified digital representation of the colored coating layer. In another example, the user may skip step (xiii) and proceed directly to step (xiii). The modified digital representation preferably includes a modified formulation obtained by transforming user input received via an interaction element. In one example, the step of providing a modified digital representation of the coating layer to the coating material manufacturing site includes providing formulation data associated with the modified digital representation to a processing device located at the coating material manufacturing site via a communication interface. In another example, the step of providing a modified digital representation of the coating layer to the coating manufacturing site includes providing formulation data associated with the modified digital representation to a computer-readable medium such as a database or cloud via a communication interface. This computer-readable medium can then be accessed by processing equipment located at the coating manufacturing site before manufacturing the coating material based on the provided modified representation. In one example, if at least one predefined tolerance, in particular all predefined tolerances, are met by the modified representation of the coating material, the modified digital representation is automatically provided to the coating manufacturer without user interaction. A message is displayed on the screen of the display device, or an email is sent to the user to inform them of the status of data transfer to the coating material manufacturing site. In another example, the user may have to actively provide the modified digital representation to the coating material manufacturing site by clicking the respective buttons on the screen of the display device.Step (xiv) may further include manufacturing a coating material based on the provided modified digital representation after the modified digital representation has been provided to the coating material manufacturing site as described above. For this purpose, the provided modified digital representation may be adapted or transformed, in particular, by a processing unit located at the coating material manufacturing site, to meet predefined manufacturing requirements or data formats necessary for the manufacturing process. The manufactured coating material can then be applied to a substrate, such as a metal plate or a dome-shaped relief, using commonly known coating techniques and heat-cured. The resulting cured colored coating layer may optionally be coated with a cured clear coat layer and then provided to the user, where it can be compared to the color designed by the user in steps (vi) through (ix). This step ensures that the color designed by the user in steps (vi) through (ix) is manufactured by the coating manufacturer based on the provided modified digital representation. If the colored coating layer does not match the user's expectations, the user may repeat steps (vi) through (ix), or the coating manufacturer may adjust the formulation of the coating material to provide a more matching colored coating layer.

[0096] Any step (xiv): If the decision made in step (xi) reveals that the modified digital representation does not meet at least one predefined tolerance, at least one recommendation may be displayed to the user on the display screen in any step (xiv). The recommendations may be stored in a computer-readable medium such as a database. For example, a computer processor may access a database containing recommendations and retrieve each recommendation based on the result of the decision. The retrieved recommendation is then displayed to the user on the display device screen. An example of a recommendation might be, "Formulation requirements are not met. Please modify the type / amount of pigment."

[0097] Step (xv): If the decision made in step (xi) reveals that the corrected digital representation does not meet at least one predefined tolerance, steps (vi) through (ix), or steps (vi) through (x), or steps (vi) through (xiv) may be repeated. It may be preferable to perform this step after displaying recommendations in order to provide user guidance on the corrections required to meet the predefined tolerances.

[0098] Steps (vi) through (ix) and any steps (x) through (xiv) allow for color correction if the appearance of the colored object displayed in step (v) does not match the desired appearance. The use of a graphic representation of the colored coating material formulation allows for a quick understanding of the effect of the coating material components on the color of the colored coating layer, thus enabling interaction guidance for color correction even without deep knowledge of the coating composition.

[0099] In one embodiment, the method further includes the step of providing an order instruction for an item to a computer processor via a communication interface, the order instruction including data relating to the color displayed in step (v), or data relating to the color associated with a modified digital representation of a colored coating layer that satisfies at least one predefined tolerance. The computer processor that receives the order instruction via the communication interface is separate from the display device, i.e., it is located in a further computing device. The further computing device comprising the computer processor may be located in a seller, such as a car dealer, or in a company that manufactures colored items, such as a car manufacturer. The seller may optionally modify the order instruction to include further data necessary for the manufacturing process or delivery to the seller, and then transfer the order instruction to the manufacturer of the item. In one example, the item may be a car. In another example, the item may be furniture, clothing, etc. The data relating to the color displayed in step (v) may be a digital representation of a colored coating layer provided in step (ii), and data associated with said digital representation, such as a color code, color name, and coating material formulation. The data relating to the color associated with a modified digital representation of a colored coating that satisfies at least one predetermined tolerance may be the coating material formulation. The order instructions may include further data such as further configuration of the item to be performed by the user, user information, and payment details. This step of the method of the present invention is preferably performed when the prediction of the item's color under different illumination conditions is performed within the process of configuring the item to be purchased, for example, within a car configurator. This allows the user to select a desired color for an item such as a car by selecting a color and viewing that color under different illumination conditions to determine whether the selected color provides the appearance desired by the user.

[0100] Embodiments of the present invention system: The aforementioned system may further include at least one database. Such a database may include a digital representation of the colored coating layer, color data related to the formulation of the coating material, a digital representation of the irradiation conditions, a model derived from past irradiation conditions, a model derived from past environmental conditions, a data-driven model parameterized to the color data of past coating layers and the past formulations of the colored coating material used to prepare past coating layers, object data of virtual objects, and color data of further coating layers. The aforementioned databases can be used in any combination.

[0101] In one embodiment, the system further comprises an apparatus for measuring color data of a colored coating layer. Suitable apparatus for measuring color data is well known in the art.

[0102] Further embodiments or aspects are described in the following numbered sections:

[0103] 1. A computer implementation method for predicting the appearance of an object coated with at least one colored coating layer and displaying the predicted appearance on a display device, wherein the method is: (i) the step of providing a digital representation of the colored coating layer to a computer processor via a communication interface; (ii) - Displaying a graphical user interface containing multiple irradiation conditions on a display device; - The computer processor detects user input indicating that an irradiation condition has been selected from the displayed multiple irradiation conditions; - In response to the detected user input, the computer processor is used via the communication interface to obtain a digital representation of the irradiation conditions associated with the detected user input; The steps include providing a digital representation of the irradiation conditions; (iii) optionally providing the computer processor via the communication interface with a model derived from past irradiation conditions and / or at least one model derived from past environmental conditions; (iv) A step of generating color data for the colored coating layer using a computer processor based on a digital representation of the provided colored coating layer, a digital representation of the provided irradiation conditions, and optionally the provided model; (v) The step of displaying the generated color data received from the computer processor on the display device, Methods that include...

[0104] 2. The method according to item 1, wherein the digital representation of the colored coating layer includes color space data, gloss data, appearance data, texture characteristics, or a combination thereof.

[0105] 3. A method for predicting the appearance of an object coated with at least one colored coating layer and displaying the predicted appearance on a display device, the method comprising the following steps: (i) Providing a digital representation of a colored coating layer to a computer processor via a communication interface, wherein the digital representation of the colored coating layer is provided by the following steps: - A step of determining the initial BTF of the colored coating layer using a camera-based measuring device; - A step of capturing spectral reflectance data of the colored coating layer for a predetermined number, i.e., a limited number of different measurement shapes, using a spectrophotometer; - To obtain the optimized BTF, the initial BTF in equation (1) is used as follows:

number

number

number

number

number

number

number

number

number

[0106] 4. The method according to any one of the preceding items, wherein the at least one colored coating layer is present on at least a portion of the surface of the object.

[0107] 5. The method according to any one of the preceding items, wherein the at least one colored coating layer is a base coat layer or a colored clear coat layer, particularly a base coat layer.

[0108] 6. The method according to any one of the preceding items, wherein the object is a motor vehicle or a part thereof.

[0109] 7. The method according to any one of the preceding items, wherein the display device is a mobile or fixed display device, preferably a mobile display device.

[0110] 8. The method according to any one of the preceding items, wherein the screen of the display device is a liquid crystal display (LCD) or an organic light-emitting diode display (OLED), and in particular an LCD or OLED display having a touchscreen panel.

[0111] 9. The method according to any one of the preceding items, wherein the display device comprises a housing for a computer processor and a screen used in steps (i) to (v).

[0112] 10. The method according to any one of items 1 to 8, wherein the display device and the computer processor that performs steps (i) to (iv) or steps (ii) to (iv) or step (iv) are configured as separate components.

[0113] 11. A method relating to any one of the preceding terms, wherein steps (iv) and (v) are performed simultaneously.

[0114] 12. The method according to any one of items 3 to 12, wherein measuring the initial BTF of the colored coating layer using a camera-based measuring device comprises using the camera-based measuring device to create a plurality of images of the colored coating layer at different viewing angles, different illumination angles, different illumination colors, and / or different exposure times, thereby providing a plurality of measurement data taking into account a plurality of combinations of illumination angle, viewing angle, illumination color, and / or exposure time.

[0115] 13. The method according to item 12, wherein images having different illumination colors and different exposure times, but having equal illumination angles and field of view, are combined into an image having a high dynamic range.

[0116] 14. In the first optimization step described above, the parameters of the first subterm for each spectral measurement geometry are optimized as follows: - From the captured spectral reflectance data, the first CIEL * a * b * Calculating a value and, - From the initial BTF of equation (1) to the second CIEL * a * b * Calculating a value and, - The first CIELa * b * From the value, the second CIELa * b * By subtracting the values, the correction vectors for the a* coordinate and b* coordinate are calculated, - Interpolate and extrapolate the correction vector for the entire range of field of view and illumination angle stored in the first sub-item, for each component. - Corrected BTF CIEL * a * b * To obtain the value, the interpolated correction vector is used for the second CIEL of each spectral measurement geometry stored in the first sub-term. * a * b *Apply to the value, - The corrected BTF CIEL * a * b * Convert the value to linear sRGB coordinates and normalize the linear sRGB coordinates, - Save the normalized sRGB coordinates to the first sub-item, The method according to any one of items 3 to 13, including

[0117] 15. The method according to item 14, wherein a multi-level B-spline interpolation algorithm is used for interpolation and extrapolation for each component of the correction vector.

[0118] 16. In the second optimization step, optimizing the parameters of the second sub-item is as follows: [[ID=Z2]]- Define a cost function based on the sum of color differences over all spectral reflectance measurement geometries, - Calculate the first CIEL * a * b * values from the captured spectral reflectance data, - Calculate the second CIEL * a * b * values from the initial BTF of formula (1) in different spectral reflectance geometries, - Using a weighted color difference formula, compare the second CIEL * a * b * values with the first CIEL * ]Fa * b * values, - Optimize the parameters of the second sub-item using a non-linear optimization method such that the cost function is minimized, The method according to any one of items 3 to 15, including

[0119] 17. The method according to paragraph 16, wherein the cost function includes a penalty function for taking into account specific constraints to keep the parameter values ​​of the intensity function within a valid range.

[0120] 18. The method according to any one of items 3 to 17, wherein the first optimization step and the second optimization step are performed repeatedly, in particular, a predetermined number of times, in order to improve the accuracy of the optimized BTF.

[0121] 19. The method according to any one of the preceding items, wherein providing a digital representation of the colored coating layer includes displaying an existing color library on the screen of the display device, selecting a color from the displayed existing library, obtaining a digital representation of the colored coating layer based on the selected color, and providing the obtained digital representation of the colored coating layer to the computer processor via the communication interface.

[0122] 20. The method according to item 19, wherein the existing color library includes at least two different colors, each corresponding to the color of a colored coating layer prepared from a coating material, and each color is defined by the optimized bidirectional texture function (BTF) or by color space data and / or gloss data and / or appearance data and / or texture properties.

[0123] 21. The method of paragraph 20, wherein displaying the existing color library on the screen of the display device includes providing object data of a virtual object, optionally providing further color data, mapping the optimized BTF or color space data and / or gloss data and / or appearance data and / or texture properties, and optionally additional color data, related to the colors present in the existing library, to the provided virtual object data, and rendering the mapping results using predefined illumination conditions.

[0124] 22. The method according to paragraph 21, wherein the virtual object is selected from virtual 2D objects such as geometric shapes, or virtual 3D objects such as chips, dome shapes, car bodies or parts thereof.

[0125] 23. The method according to item 21 or 22, wherein the predefined illumination conditions are selected from a direct light source or a high dynamic range (HDR) environment map, in particular a high dynamic range (HDR) environment map.

[0126] 24. The method according to any one of claims 1 to 18, wherein the step of providing a digital representation of the colored coating layer includes providing coating layer identification data, obtaining a digital representation of the coating layer based on the provided coating layer identification data, and providing the obtained digital representation.

[0127] 25. The method of item 24, wherein providing coating layer identification data includes providing the optimized bidirectional texture function (BTF) of the colored coating layer and / or providing data indicating the colored coating layer.

[0128] 26. The method according to any one of paragraphs 19 to 25, wherein the step of obtaining a digital representation of the colored coating layer is further defined as searching a database of digital representations based on the selected color or identification of the provided coating layer.

[0129] 27. The method according to any one of the preceding items, further comprising step (i) displaying a color relating to the digital representation of the provided colored coating layer on the screen of the display device.

[0130] 28. The method according to any one of the preceding claims, wherein the plurality of illumination conditions includes at least (i) ambient illumination surrounding the display device, and (ii) a predefined high dynamic range (HDR) environment map, and (iii) ambient light associated with a specific time, date, and location.

[0131] 29. The method according to any one of the preceding claims, wherein the user input is detected via an interaction element, particularly a physical interaction element such as a mouse, keyboard, trackball, touch screen, or a combination thereof.

[0132] 30. Obtaining a digital representation of the illumination conditions comprises - obtaining, via the communication interface, data indicating a date, time, location, particularly a geographical location, and optionally a degree of clear sky haze, and / or - obtaining, via the communication interface, data obtained from at least one illumination sensor of the display device, data obtained from at least one orientation sensor of the display device, and optionally data obtained from at least one viewpoint sensor, and / or - obtaining at least one high dynamic range (HDR) environment map via the communication interface, The method according to any one of the preceding claims.

[0133] 31. The step of obtaining, via the communication interface, data indicating a location, particularly a geographical location, includes displaying a world map on the display device, detecting a user input indicating selection of a location on the displayed world map via the interaction element, and obtaining, in response to the detected user input, data associated with the selected geographical location, particularly GPS data, via the communication interface. The method according to claim 30.

[0134] 32. Obtaining data indicating the date and / or time and / or degree of empty haze via the communication interface includes displaying at least one adjustment tool including at least one regulator corresponding to the date or time or haze, and detecting a user input indicating operating the at least one adjustment tool via an interaction element, particularly detecting a user input by detecting the movement of at least one regulator of the at least one adjustment tool via the interaction element, and determining the date or time or haze associated with the position of each regulator in response to the detected user input. The method according to item 30 or 31.

[0135] 33. The data obtained from at least one irradiation sensor of the display device includes data regarding the irradiation conditions surrounding the display device, such as lux level, spectral components, irradiation direction, at least one photograph of the environment surrounding the display device, particularly at least one high-dynamic range (HDR) or low-dynamic range (LDR) photograph, or a combination thereof. The method according to any one of items 30 to 32.

[0136] 34. The data obtained from at least one viewpoint sensor of the display device includes at least one photograph showing a user viewing the display device. The method according to any one of items 30 to 33.

[0137] 35. Obtaining at least one high-dynamic range (HDR) environment map includes displaying at least one high-dynamic range (HDR) environment map on the display device, detecting a user input indicating selection of the high-dynamic range (HDR) environment map displayed via the interaction element, and obtaining the high-dynamic range (HDR) environment map associated with the detected user input via the communication interface in response to the detected user input. The method according to any one of items 30 to 34.

[0138] 36. The method according to any one of the preceding items, further comprising step (ii) displaying the provided irradiation conditions on the screen of the display device.

[0139] 37. A model derived from past irradiation conditions is a physically based analytical model of the daytime sky, as described in any one of the preceding terms.

[0140] 38. The method according to any one of the preceding items, wherein at least one model derived from the past environmental conditions provides a relationship between data obtained from the illumination sensor and / or orientation sensor and / or viewpoint sensor of the display device and the ambient light conditions surrounding the display device.

[0141] 39. The method according to item 38, wherein the data acquired by the illumination sensor and / or the viewpoint sensor includes high dynamic range (HDR) photographs or low dynamic range (LDR) photographs.

[0142] 40. The method according to any one of the preceding items, wherein the step of generating color data with the computer processor includes providing object data of a virtual object; optionally providing further color data for at least one further coating layer; mapping the digital representation of the provided colored coating layer and the optional further color data onto the provided virtual object; and rendering the mapping result using the digital representation of the provided illumination conditions and optionally the provided model.

[0143] 41. The method according to paragraph 40, wherein the virtual object is a 2D virtual object such as a geometric shape, or a 3D virtual object such as a chip, a dome shape, a car body or a part thereof.

[0144] 42. The method according to any one of the preceding items, further comprising, step (iv), calculating the ambient illumination conditions surrounding the display device using the computer processor from a digital representation of the provided illumination conditions and at least one model derived from the provided historical environmental conditions, before generating the color data for the colored coating layer.

[0145] 43. Using the computer processor to calculate the ambient illumination conditions surrounding the display device is: - Calculating ambient light conditions by estimating the brightness and color of the data acquired by the irradiation sensor, - Calculating a high dynamic range (HDR) environment map from the illumination sensor and the data acquired by the illumination sensor, or - Calculate spherical harmonic (SH) coefficients from the data acquired by the illumination sensor and optionally from the data acquired by the viewpoint sensor, and optionally calculate an HDR environment map from the calculated SH coefficients. The method described in paragraph 42, including the method described in paragraph 42.

[0146] 44. The method according to any one of the preceding items, wherein the generated color data is recalculated in response to a change in data included in the digital representation of the illumination conditions provided in step (iii), in particular in response to a change in data provided from at least one illumination sensor and / or in response to a change in data provided from at least one orientation sensor and / or in response to a change in data provided from at least one viewpoint sensor, and / or in response to a change in data provided from at least one viewpoint sensor, and displayed on the screen of the display device.

[0147] 45. The method according to any one of the preceding items, wherein the step of displaying the color data received from the computer processor on the screen of the display device includes mapping the generated color data, in particular each rendered point, to the screen of the display device.

[0148] 46. ​​The method according to any one of claims 27 to 45, wherein the step of displaying the color data received from the processor on the display device includes automatically updating the color displayed on the screen of the display device in step (i) in response to performing step (v).

[0149] 47. The method according to paragraph 46, wherein the step of automatically updating the colors displayed on the display device includes updating the colors displayed in step (i) with the generated color data, in particular by mapping each rendering point to the display device, the generated color data generated in step (iv).

[0150] 48. The method according to any one of the preceding clauses, further comprising step (v) storing the generated color data and / or illumination conditions, the provided digital representation and further data such as ambient illumination conditions calculated from at least one provided model, in a computer-readable medium.

[0151] 49. The method according to any one of the preceding items, further comprising step (v) adding the displayed colors to an existing color library or a newly generated color library to generate a modified color library.

[0152] 50. The modified color library is associated with a saved user profile for future use when designing the colors of the colored coating layer, as described in Section 49.

[0153] 51. The method of Clause 49 or 50, further comprising removing at least one additional color from the modified color library.

[0154] 52. To create a sorted color library, sorting the colors existing in an existing color library or a modified color library according to grouping criteria, and optionally associating the sorted color library with a saved user profile for future provisioning of the sorted color library, the method according to any one of clauses 49 to 51 further comprising.

[0155] 53. The step of displaying the generated color data on the screen of the display device includes automatically updating the color displayed on the screen of the display device in step (v) in response to repeating steps (ii) to (v), or in response to repeating steps (iii) to (v), or in response to repeating steps (ii) and (v), the method according to any one of the preceding clauses.

[0156] 54. The method according to any one of the preceding clauses further comprising repeating steps (i) to (v), or steps (ii) to (v), or steps (i), (iv) and (v).

[0157] 55. (vi) Modifying the digital representation of the provided colored coating layer to generate a modified digital representation of the colored coating layer; (vii) Optionally repeating step (vii); (viii) Generating color data of the coating layer based on the modified digital representation of the colored coating layer; (ix) Displaying the generated color data received from the processor on the screen of the display device; (x) Optionally repeating steps (vii) to (ix); (xi) Optionally, determining by the computer processor whether the modified digital representation of the colored coating layer is within at least one predefined tolerance. (xii) Optionally, according to the determination that the modified digital representation is within at least one predefined tolerance: the steps (ii) through (v) are repeated using the modified digital representation of the colored coating layer; (xiii) optionally, in accordance with the determination that the modified digital representation is within at least one predefined tolerance: providing the modified digital representation to a coating material manufacturing site via the communication interface; optionally, manufacturing a coating material based on the provided modified digital representation; (xiv) Optionally, in accordance with the determination that the modified digital representation is outside of at least one predefined tolerance: the step of displaying at least one recommendation on the screen of the display device; (xv) Optionally, in accordance with the determination that the modified digital representation is outside at least one predefined tolerance: a step repeating step (vi) to (ix), or step (vi) to (x), or step (vi) to (xiv), A method of any one of the preceding items, further including the method described in any one of the preceding items.

[0158] 56. The method according to any one of the preceding items, further comprising the step of providing an order instruction for an item to a computer processor via the communication interface, wherein the order instruction includes data relating to the color displayed in step (v), or data relating to the color relating to the modified digital representation of the colored coating layer that satisfies at least one predefined tolerance.

[0159] 57. A system for predicting the appearance of an object coated with at least one colored coating layer, wherein the system: - Optionally, a communication interface for providing a computer processor with a model derived from past irradiation conditions and / or at least one model derived from past environmental conditions; - At least one communication interface for providing a computer processor with a digital representation of the colored coating layer and a digital representation of the irradiation conditions; - A display device equipped with a screen; - Interaction elements for detecting user input; - Optionally, at least one illumination sensor and / or at least one orientation sensor adapted to sense the orientation of the display device and / or at least one viewpoint sensor adapted to sense the viewpoint of a user holding the display device; - A processor that communicates with the communication interface, the display device, and the at least one illumination sensor and / or orientation sensor and / or viewpoint sensor, wherein the processor: • Receive the digital representation of the colored coating layer via the communication interface; • Generate a user interface presentation containing multiple irradiation conditions, detect user input indicating that an irradiation condition has been selected from the displayed multiple irradiation conditions, and in response to the detected user input, retrieve a digital representation of the irradiation condition associated with the detected user input; Optionally, calculate the ambient illumination conditions surrounding the display device from the digital representation of the acquired illumination conditions and the received model; Based on the received digital representation of the colored coating layer and the received digital representation of the illumination conditions or the calculated ambient illumination conditions surrounding the display device, the color data of the colored coating layer is generated. The processor is programmed to do so, Equipped with, The display device receives the generated user interface presentation and the generated color data of the colored coating layer from the processor, and displays the generated user interface presentation and color data. The digital representation of the aforementioned colored coating layer is performed in the following steps: - A step of determining the initial BTF of the colored coating layer using a camera-based measuring device; - A step of capturing spectral reflectance data of the colored coating layer for a predetermined number, i.e., a limited number of different measurement shapes, using a spectrophotometer; - To obtain the optimized BTF, the initial BTF in equation (1) is used as follows:

number

number

number

number

number

number

number

number

number

[0160] 58. A system for predicting the appearance of an object coated with at least one colored coating layer, wherein the system: - Optionally, a communication interface for providing a computer processor with a model derived from past irradiation conditions and / or at least one model derived from past environmental conditions; - At least one communication interface for providing the computer processor with a digital representation of the colored coating layer and a digital representation of the irradiation conditions; - A display device equipped with a screen; - Interaction elements for detecting user input; - Optionally, at least one illumination sensor and / or at least one orientation sensor adapted to sense the orientation of the display device and / or at least one viewpoint sensor adapted to sense the viewpoint of a user holding the display device; - A first processor that communicates with the communication interface, the display device, and the at least one illumination sensor and / or orientation sensor and / or viewpoint sensor, wherein the first processor: • Generate a user interface presentation containing multiple irradiation conditions, detect user input indicating that an irradiation condition has been selected from the displayed multiple irradiation conditions, and in response to the detected user input, retrieve a digital representation of the irradiation condition associated with the detected user input; Optionally, calculate the ambient illumination conditions surrounding the display device from the digital representation of the acquired illumination conditions and the received model. The first processor is programmed in such a way, - A second processor that communicates with the communication interface and the first processor, wherein the second processor is: The digital representation of the colored coating layer and the digital representation of the illumination conditions or ambient illumination conditions calculated by the first processor are received via the communication interface; Based on the received digital representation of the colored coating layer and the received digital representation of the irradiation conditions or the calculated ambient irradiation conditions received from the first processor, color data of the colored coating layer is generated. The second processor is programmed to do so, Equipped with, The display device receives a user interface presentation generated from the first processor and color data generated from the second processor. The digital representation of the aforementioned colored coating layer is performed in the following steps: - A step of determining the initial BTF of the colored coating layer using a camera-based measuring device; - A step of capturing spectral reflectance data of the colored coating layer for a predetermined number, i.e., a limited number of different measurement shapes, using a spectrophotometer; - To obtain the optimized BTF, the initial BTF in equation (1) is used as follows:

number

number

number

number

number

number

number

number

number

[0161] 59. A system for predicting the appearance of an object coated with at least one colored coating layer, the system comprising: a display; one or more computing nodes; and one or more computer-readable storage media having computer-executable instructions structured to cause the system to perform the method described in any one of items 1 to 56 when executed by the one or more computing nodes.

[0162] 60. A non-transient computer-readable storage medium, wherein the computer-readable storage medium includes instructions, when executed by a computer, causing the computer to perform steps according to any one of the methods described in items 1 to 56.

[0163] 61. Use of the method described in any one of items 1 to 56 or the system described in any one of items 57 to 59 for predicting the appearance of an object coated with at least one colored coating layer.

[0164] 62. An object coated with at least one colored coating layer, wherein the color of at least one coating layer is predicted according to the method described in any one of items 1 to 56. [Brief explanation of the drawing]

[0165] These and other features of the present invention are described more fully in the following description relating to exemplary embodiments of the invention. To facilitate identification of any particular element or action, the most significant digit of the reference number refers to the figure number in which that element is first introduced. This specification is presented with reference to the accompanying drawings: [Figure 1] Figure 1a is a block diagram of a method for predicting the appearance of an object coated with at least one colored coating layer, according to a first embodiment of the present invention. Figure 1b is a block diagram of a method for predicting the appearance of an object coated with at least one colored coating layer, according to a second embodiment of the present invention. [Figure 2] Figure 2a shows a system according to the first embodiment of the present invention. Figure 2b shows a system according to the second embodiment of the present invention. Figure 2c shows a system according to the third embodiment of the present invention. [Figure 3] This is a plan view of a system comprising a display device having a graphical user interface that shows an existing color library in which a colored coating layer is displayed on a virtual 3D object in the form of a dome-shaped relief. [Figure 4] This is a plan view of a system comprising a display device having a graphical user interface showing adjustment tools and a virtual 3D object in the shape of a car body colored with colors related to the selected illumination conditions. [Figure 5]Figure 5a is a 3D view of a system comprising a display device having a graphical user interface that shows virtual 3D objects in the form of dome-shaped reliefs colored with colors related to ambient illumination and the orientation of the display device. Figure 5b is a 3D view of a system comprising a display device having a graphical user interface that shows virtual 3D objects in the form of dome-shaped reliefs colored with colors related to ambient illumination and the orientation of the display device. [Figure 6] This is a plan view of a system equipped with a display device having a graphical user interface that shows the available high dynamic range (HDR) map environment. [Figure 7] Figure 7a is a plan view of a system comprising a display device having a graphical user interface that shows a virtual 3D object in the form of a part of an automobile body colored using a selected high dynamic range (HDR) map environment. Figure 7b is a plan view of a system comprising a display device having a graphical user interface that shows a virtual 3D object in the form of a part of an automobile body colored using a selected high dynamic range (HDR) map environment when viewed from different viewing angles. [Modes for carrying out the invention]

[0166] Detailed description of the drawing The detailed description below is intended to illustrate various aspects of the subject matter and not to represent the only possible configurations in which the subject matter may be implemented. The accompanying drawings are incorporated herein and constitute part of the detailed description. The detailed description includes specific details for the purpose of providing a complete understanding of the subject matter. However, it will be apparent to those skilled in the art that the subject matter can be implemented without these specific details.

[0167] Figure 1a shows a non-limiting first embodiment of Method 100 according to the present invention for predicting the appearance of an object coated with at least one colored coating layer and displaying the predicted appearance on the screen of a display device. In this example, the colored coating layer is a base coat layer, and the object is an automobile or a part thereof, such as a trim part or bumper. In this example, the display device is a portable display device having an LCD screen with a touchscreen, such as a tablet or smartphone. In another example, the display device is a fixed device such as a fixed computer, a television screen connected to a computer via a communication interface, or a projection display device. In this example, the display device has a housing that houses the screen as well as a processor. In yet another example, the processor resides separately from the display device on an external device coupled to the display device, for example, via a wired or wireless communication interface.

[0168] In block 102, routine 101 decides whether to display an existing color library on the display device screen. This decision can be made based on user data, such as a user profile. This allows different existing color libraries to be displayed depending on the user, so that each existing color library can be customized to enhance user comfort. If routine 101 decides in block 102 to display an existing color library, it proceeds to block 104; otherwise, it proceeds to block 106, which will be described later.

[0169] In block 104, routine 101 displays each existing color library on the display device screen. In this example, the existing library contains 195 different colors, each corresponding to the color of a colored coating layer prepared from a colored coating material, and each color is defined by an optimized BTF as described above. In this example, the predefined color library provides predefined object data for virtual objects, and these are displayed on the display device screen by rendering the color data present in the predefined color library and the provided object data using predefined illumination conditions (image-based lighting) (see, for example, Figure 3). The predefined object data for virtual objects is provided to the processor from a database via a communication interface. In this example, the virtual object is a 3D virtual object with a dome shape. In another example, the virtual object is a 2D virtual object, such as a colored region. In this example, the user can scroll through the displayed existing color library to see all the colors displayed via an interaction element. The interaction element may be a touchscreen gesture, a mouse click, a shortcut, or a combination thereof.

[0170] In block 106, routine 101 detects user input (e.g., the user selects a color from an existing library displayed), retrieves a digital representation of the colored coating layer from the database based on the detected user input (e.g., the selected color), and provides the retrieved digital representation to the computer processor via a communication interface. The digital representation of the colored coating layer includes an optimized bidirectional function (BTF) obtained in the steps described above and may include further data such as the formulation of the coating material used to prepare the colored coating layer, the color name, and the color identification number. User input can be performed via interaction elements such as touchscreen gestures, mouse clicks, or shortcuts.

[0171] In block 108, routine 101 displays multiple illumination conditions on the screen of a display device, for example, within a GUI. For this purpose, routine 101 can generate a user interface presentation including the multiple illumination conditions, which is then displayed on the screen of the display device. In this example, the available illumination conditions include the use of ambient light surrounding the display device (environment), the use of an existing HDR environment map (HDRI), and the use of ambient light associated with a specific time, date, and location (sky model). The multiple illumination conditions may be displayed within the user interface using icons, images, text, or a combination thereof, and may be selected by selecting each icon, image, text, or a combination thereof via an interaction element. In one example, the selected illumination conditions are displayed to the user, and the GUI may provide the possibility to return to the selection of different illumination conditions to allow user input to be modified.

[0172] In block 110, routine 101 detects user input indicating that an irradiation condition has been selected from the displayed multiple irradiation conditions, and proceeds to block 112.

[0173] In block 112, routine 101 determines the illumination conditions selected by the user via an interaction element, based on the user input detected in block 110. Depending on the user selection, routine 101 proceeds to block 114 (if the determined user selection corresponds to the use of an HDR environment map), block 116 (if the determined user selection corresponds to the use of ambient light surrounding the display device), or block 124 (if the determined user selection corresponds to the use of ambient light associated with a specific time, date, and location).

[0174] In block 114, routine 101 displays the available HDR environment maps on the display device's screen, for example, within a GUI. In this example, the available HDR environment maps are displayed on the GUI by retrieving images associated with the available environment maps from a data storage medium such as a database or the display device's internal memory, and displaying the retrieved images (for example, as shown in Figure 6 below). The user's selection of a displayed HDR image via an interaction element is detected by routine 101 and used to retrieve the respective HDR environment map, as described below in block 116. In one example, the user can further select the orientation of the selected HDR environment map, as well as the viewing direction used to generate color data in block 134. For this purpose, routine 101 displays a GUI containing the selected HDR environment map, and the user can change the orientation by rotating the map. The viewing direction may be selected by displaying a list of available directions or by displaying graphics including objects and people, and the user can select the appropriate entry from the list or by moving the person relative to the displayed object. The user's selection is detected by routine 101 and translated by the computer processor into the selected orientation and / or selected viewing direction of the HDR environment map. In another example, a predefined orientation and predefined viewing direction of the HDR environment map are used to generate color data in block 134, and the user can change the orientation and / or viewing direction of the HDR environment map, as described later.

[0175] In block 116, routine 101 retrieves the HDR environment map associated with the user selection detected in block 114 from a data storage medium such as a database, and provides the retrieved HDR environment map to the processor. The retrieved HDR environment map corresponds to the digital representation of the illumination conditions described above.

[0176] In block 118, routine 101 initiates the acquisition of data from the illumination and / or orientation and / or viewpoint sensors of the display device and stores the acquired data in a data storage medium such as the internal memory of the display device. Data acquisition may be performed over a predetermined period of time, or until the user terminates method 100 or selects different illumination conditions as described later. The data may be stored along with further metadata such as acquisition time and sensor name. If method 100 is repeated several times, the acquired data may be deleted at the end of the method to reduce the amount of data storage required to store the acquired sensor data. The data acquired by the illumination and / or orientation and / or viewpoint sensors of the display device corresponds to the digital representation of the illumination conditions described above.

[0177] In block 120, routine 101 retrieves models derived from past environmental conditions from a data storage medium such as a database or the internal memory of a display device via a communication interface, and provides the retrieved models to a computer processor. Appropriate models derived from past environmental conditions include the aforementioned HDRI model, ambient light model, and spherical harmonic coefficient model, as well as a data-driven model that estimates the HDR environment map from the SH coefficient as described above.

[0178] In block 122, ambient illumination conditions are calculated based on the model acquired in block 120 or block 126 (described later) and the sensor data acquired in block 118 or the user input detected in block 124 (described later). In one example, calculating ambient illumination conditions may include calculating ambient light conditions from color and brightness estimated from data acquired by the illumination sensor, such as at least one photograph of the environment surrounding the display device, as described above. In another example, calculating ambient illumination conditions may include calculating an HDR environment map and / or spherical harmonic (SH) coefficients from the acquired sensor data, as described above. The calculated ambient illumination conditions may be displayed on the screen of the display device, for example, in the form of a calculated HDR environment map (see, for example, Figure 4, reference numeral 416).

[0179] In block 124, routine 101 detects user input for date, time, location, and optionally the degree of haze. In this example, a GUI including at least one adjustment tool that allows the user to input at least one of the date, time, location, and optionally an empty degree of haze is displayed on the screen of a display device, for example, as shown in Figure 4. The user can input the date, time, location, and optionally the degree of haze via interaction elements using the displayed adjustment tool and further icons such as an interactive calendar or map. The displayed adjustment tool or icon may include predefined data such as the current time, current date, location of the display device, and a predefined degree of haze. The regulator of the adjustment tool can be operated by the user via interaction elements. Routine 101 can determine the time, date, and optionally the haze from the location of the regulator of the adjustment tool. The location of the display device can be determined using the GPS module of the display device. The input or predefined data regarding the time, date, location, and optionally the degree of haze corresponds to the digital representation of the illumination conditions described above.

[0180] In block 126, routine 101 retrieves a model derived from past irradiation conditions from a data storage medium such as a database or the internal memory of a display device via a communication interface, and provides the retrieved model to the computer processor. The appropriate model derived from past irradiation conditions is the daytime sky physics-based analysis model described in I. Hosek et al., "An analytic model for full spectral sky-dome radiance," ACM Transactions on Graphics, 2012, Vol. 31, Article No.: 95, https: / / doi.org / 10.1145 / 2185520.2185591. After block 126, routine 101 proceeds to block 122 as described above.

[0181] After block 116 or 122, routine 101 proceeds to block 128. In block 128, routine 101 determines whether predefined virtual object data is used to generate color data, which will be described later in relation to block 134. If predefined virtual object data is used, routine 101 proceeds to block 130; otherwise, routine 101 proceeds to block 132.

[0182] In block 130, routine 101 retrieves predefined virtual object data. In this example, this includes displaying icons, images, text, or a combination thereof on the display device screen that represent available virtual objects on the GUI; detecting user input indicating the selection of a displayed virtual object; retrieving virtual object data from a data storage medium such as a database or the display device's internal memory based on the detected user input; and providing the retrieved virtual object data to the computer processor. Available virtual objects may include 2D objects such as colored regions, or 3D objects such as dome-shaped reliefs or different types of common automobiles (i.e., shapes of automobiles representing different types of automobiles such as SUVs and sports cars).

[0183] In block 132, routine 101 detects user-provided virtual object data, for example, by providing a file containing virtual object data, or by selecting each file stored in the internal memory of the display device. This may include providing a GUI on the display device screen where the available options for providing virtual object data are displayed to the user. By using user-defined virtual object data, the virtual object data can be customized to the user's specific needs, thus improving user comfort.

[0184] In one example (not shown), method 100 may further include block 133. In block 133, routine 101 determines whether at least one further colored coating layer is used to generate color data in block 134. If so, routine 101 provides the processor with color data for at least one further colored coating layer. Otherwise, routine 101 proceeds to block 134, which is described below. In one example, the color data for at least one further coating layer is provided by displaying an existing color library, detecting the user's selection of a display color, retrieving color data related to the selected color from a data storage medium such as a database or the internal memory of the display device, and providing the retrieved color data to the computer processor. In another example, the user can input information about the type of further colored coating layer (e.g., further base coat, colored clear coat) and / or color information (e.g., color name, color code), and routine 101 retrieves color data based on the input information and provides the retrieved color data to the computer processor. By using color data from at least one additional colored coating layer, color data for a multilayer coating containing two or more colored coating layers can be generated, and thus method 100 can be adapted to the respective layer structure of the coating whose appearance is predicted.

[0185] In block 134, the color data for the coating layer is: - Digital representation of the colored coating layer provided in Block 106 and - Virtual object data provided in block 130 or 132 and - The HDR environment map provided in block 116, or the ambient illumination conditions calculated in block 122, It is generated using a computer processor based on this.

[0186] When color data is generated using the HDR environment map provided in block 116, the orientation and / or viewing angle of the HDR environment map selected in block 114, or the predefined orientation and / or predefined viewing direction of the HDR environment map, is used in the rendering process. When the predefined orientation of the HDR environment map is used, the orientation can be adapted by the user by rotating the HDR environment map displayed in blocks 138 / 140, as described later. Color data is generated by mapping a digital representation of the provided colored coating layer onto a provided virtual object and rendering the mapping result using the provided HDR environment map or provided ambient illumination conditions. Rendering using the provided HDR environment map or provided ambient illumination conditions is performed, for example, using image-based lighting (IBL) as described above.

[0187] In block 136, routine 101 determines whether the color data associated with the digital representation of the provided colored coating layer is already displayed on the screen of the display device. For example, the color data associated with the digital representation of the provided colored coating layer may be displayed in block 104 or block 124. If the color data associated with the digital representation of the provided colored coating layer is already displayed on the screen of the display device, routine 101 proceeds to block 140; otherwise, routine 101 proceeds to block 138.

[0188] In block 138, routine 101 displays the color data generated in block 134 on the display device screen by mapping the generated color data, i.e., each rendered point, to the display device screen, causing the rendered object to be displayed on the display device screen, positioned in front of a predefined or selected background. In this example, the rendered object is positioned in front of a predefined background with a uniform color. In another example, the rendered object is positioned in front of a user-defined background. The user-defined background may be provided by detecting a background selection from a list of predefined backgrounds, or by using a background provided by the user, such as a photograph. Using a user-defined background allows for customization of the surroundings of the virtual object, giving the user an impression of how the object will look in a particular environment. This allows the virtual object to be placed in a real-world environment, potentially improving the user's ability to determine whether the displayed appearance meets their requirements. The color data may be displayed in a GUI including additional icons, images, text, or combinations thereof, allowing the user to perform further actions such as changing the viewing direction in the HDR environment map (see, for example, Figure 7b), saving the generated color data, adding comments to the generated color data, or executing further blocks described later. In addition to color data, further data such as ambient illumination conditions calculated in block 122, adjustment tools displayed in block 124, or information about the selected color in block 106 may be displayed on the screen of the display device.

[0189] In block 140, routine 101 automatically updates the colors displayed on the display device's screen using the color data generated in block 134, by mapping the generated color data, i.e., each rendered point, to the display device's screen, so that previously displayed colors are replaced by the generated color data. The update may include displaying additional icons, images, text, or combinations thereof, or additional data on the display device's GUI, as described in relation to block 138.

[0190] In block 142, routine 101 determines whether the sensor data acquired after block 138 or block 140 has been modified compared to the sensor data acquired in block 118, or whether the user has modified input data such as the data entered in block 124, or predefined data such as the field of view of the HDR environment map selected in block 114. If routine 101 determines that the sensor data, user input, or predefined data has not been modified, the routine proceeds to block 144, which will be described later. If routine 101 determines that the sensor data, user input, or predefined data has been modified, routine 101 proceeds to either block 122 (if the sensor data or user input has been modified) or block 134 (if the predefined data has been modified by the user).

[0191] Determining a change in sensor data may involve comparing the data acquired in block 118 with the data acquired after executing block 122 and determining whether the difference exceeds a predefined threshold. If data acquisition is completed before executing block 122, routine 101 may start acquiring sensor data as described in block 142 in relation to block 122 and compare the data acquired in block 118 with the data acquired in block 142 as described above. If routine 101 determines that the difference in the acquired sensor data exceeds a predefined threshold, routine 101 proceeds to block 122, where the processor calculates the ambient illumination conditions using the sensor data acquired after block 122, or the sensor data acquired in block 142 as described above in relation to block 122. If routine 101 determines that the difference in the acquired sensor data is less than a predefined threshold (i.e., no significant change has occurred), routine 101 proceeds to block 144.

[0192] Determining a change in user input may include, for example, comparing the data entered in block 124 with the respective data entered in block 138 by moving the regulator of an adjustment tool or interactive icon displayed on the GUI in block 138.

[0193] Deciding to change predefined data may involve comparing the viewing angle of a predefined HDR environment map with the viewing angle of the HDR environment map displayed in block 138. If the user changes the viewing angle in block 138, routine 101 proceeds to block 134 and generates color data using the HDR environment map provided in block 116 and the viewing angle defined in block 138.

[0194] As described above, the generated color data can be automatically updated using the current sensor data, user input, or predefined data by recalculating and displaying the color data on the display device's screen in response to changes in acquired sensor data, user input, or predefined data. In one example, the recalculation and display of the regenerated color data is performed automatically, i.e., without user interaction, when the aforementioned changes in acquired, input, or predefined data are detected. This allows the user to understand the effect that various parameters, such as the orientation, location, time, data, degree of haze, or viewing angle of the display device, have on the displayed appearance. Since the recalculation and display of the recalculated color data is performed in real time or near real time, when the user tilts or rotates the display device or changes the viewpoint or time / date / location / haze, the user can get the impression that the appearance of the colored object displayed on the screen behaves like a real sample.

[0195] In block 144, routine 101 decides whether to change the illumination conditions determined in block 112 or to exit method 100. In one example, this includes displaying the respective menus on the screen of a display device and prompting the user to select the desired option. In another example, routine 101 detects user input indicating that the user wants to return to block 110, for example, by detecting the selection of each user icon indicating the available illumination conditions. If routine 101 decides in block 144 that the user wants to change the illumination conditions, routine 101 proceeds to block 110. Otherwise, routine 101 proceeds to block 146 or exits method, depending on the user's selection. By returning to block 110, the user can examine the effect of different illumination conditions on the selected color and determine whether the selected color has the desired appearance under all illumination conditions related to the colored object.

[0196] In block 146, routine 101 determines whether to change the color selected in block 104. In one example, this involves displaying each menu on the display device's screen prompting the user to select a desired option. In another example, routine 101 detects user input indicating that the user wants to return to block 102, for example, by detecting the selection of each user icon indicating that a color has been selected. In block 146, if routine 101 determines that the user wants to change the color, routine 101 proceeds to block 102. Otherwise, routine 101 terminates the method. Returning to block 102, the user can select a new color if the appearance of the color selected in block 104 is not the desired appearance.

[0197] Figure 1b shows a non-limiting second embodiment of method 100' according to the present invention for predicting the appearance of an object coated with at least one colored coating layer and displaying the predicted appearance on the screen of a display device. In this example, the colored coating layer is a base coat layer, and the object is an automobile or its parts, such as a trim part or bumper. In this example, the display device is a portable display device having an LCD screen, including a touchscreen, such as a tablet or smartphone. In another example, the display device is a fixed device, such as a fixed computer, a television screen connected to the computer via a communication interface, or a projection display device. In this example, the display device has a housing that houses the screen as well as a processor. In yet another example, the processor resides separately from the display device, on an external device coupled to the display device, for example, via a wired or wireless communication interface.

[0198] Method 100' in Figure 1b includes blocks 102 to 146, which were previously described in relation to Figure 1a. Furthermore, Method 100' includes blocks 148 to 168, which are described below. In one example, Method 100' is performed when the user determines that the predicted appearance of a colored object displayed on the screen of the display device in block 138 or block 140 does not match the appearance desired by the user n. The user can then perform Method 100' to correct the color displayed on the screen of the display device in block 138 or block 140. In another example, Method 100' is performed when the user wishes to place an order for an item to be colored with the color selected in block 104.

[0199] In block 148, routine 101' determines whether the visual appearance of the color data displayed in block 138 or 140 is acceptable. This includes displaying the respective menus on the GUI and prompting the user to make the appropriate selection. If routine 101' determines in block 148 that the visual appearance is acceptable, it proceeds to block 150; otherwise, it proceeds to block 154, which will be described later.

[0200] In block 150, routine 101' determines what further action the user desires. This may include displaying the respective dialogs on the GUI and prompting the user to make the appropriate selection. If routine 101' determines that the user desires to repeat the aforementioned block 102 or 110 (i.e., change the illumination conditions or color), routine 101' proceeds to block 102 or 110, respectively. If routine 101' determines that the user desires to end the method, routine 101' terminates method 100'. If routine 101' determines that the user desires to place an order, routine 101' proceeds to block 152, described below. By proceeding to block 110 using the modified digital representation of the colored coating layer, the user can verify whether the modified color yields the desired appearance under the available illumination conditions, thus improving user comfort during the virtual color design process. In this block, the user may also have the option to optionally save the generated color data, along with additional data such as comments and labels, to a data storage medium such as a database or the internal memory of the display device. This allows us to retrieve stored color data as needed, and therefore reduce the time required to calculate each color data.

[0201] In block 152, routine 101' generates an order instruction for an item and provides the generated order instruction to a further computing device via a communication interface. In one example, the further computing device is located at a seller such as a car dealer, or a company that manufactures colored items such as a car manufacturer, or a coating material manufacturing site. The provided order instruction includes data relating to the color shown in block 138 or 140, such as the color code, color name, formulation of the coating material used to prepare the colored coating layer, the (modified) formulation of the coating material, and combinations thereof. The order instruction may include further data such as further configuration of the item to be executed by the user, user information, and payment details. By sending an order instruction for an item having a desired color within method 100' of the present invention, the user can directly provide the seller or manufacturer with all the necessary details regarding the desired color, thus avoiding unnecessary data transfer and reducing the risk of data loss or erroneous data transfer. After sending the order instruction, the colored item / coating material may be manufactured based on the information contained in the provided order and provided to the user associated with the provided order. Routine 101' then proceeds to block 153.

[0202] In block 153, routine 101' determines whether the user wants to exit method 100' or return to block 102, as described in relation to Figure 1a. For this purpose, routine 101' can display the respective dialogs on the display device screen to prompt the user for the appropriate choice. If routine 101' determines that the user wants to exit method 100', routine 101' exits method 100'. Otherwise, routine 101' proceeds to block 102, as described in relation to Figure 1a.

[0203] In block 154, routine 101' displays a digital representation of the colored coating layer provided in block 106 on the screen of the display device. The displayed digital representation includes at least some of the components used to prepare the coating material associated with the provided digital representation; that is, at least some of the components present in the coating material used to prepare the color selected in block 104 are displayed on the screen of the display device in block 154. In one example, a rendered virtual object obtained by rendering a virtual object with the color data contained in the digital representation provided in step 106 using predefined illumination conditions as described above is additionally displayed on the screen in this block. In this example, predefined virtual object data of a part of a car is used, and the displayed virtual object can be rotated or zoomed. In another example, the user selects a virtual object, and object data associated with the selected object is retrieved from a database and used in the rendering process.

[0204] In block 156, routine 101' detects user modification of a displayed digital representation of a colored coating layer. This may include the user modifying at least one displayed component via an interaction element such as a touchscreen gesture, and detecting user input using a display device processor. In this example, the components present in the coating formulation used to prepare the colored coating layer are represented using at least one adjustment tool comprising multiple regulators, each regulator corresponding to the type and amount of component present in the coating material. The adjustment tool is generated by routine 101' from the digital representation of the colored coating layer provided in block 106 by converting data relating to the coating material formulation contained in the digital representation into an adjustment tool. In this example, the adjustment tool is a box having multiple visually distinguishable compartments, each compartment representing a type of component present in the coating material, the size of each compartment representing the amount of each component, and the multiple regulators corresponding to lines separating the compartments. The type of component is displayed in each compartment using a graphical representation indicating the type of each component. The graphical representation is obtained from the digital representation of the provided colored coating layer by associating the formulation data contained in the provided digital representation with predefined images for each type of component, such as color chips for coloring pigments, metal pieces for metallic effect pigments, glass pieces for glass flakes, and solid blocks for binders, and displaying the images in their respective sections. Using boxes with sections of different sizes that contain graphical representations of specific components allows for easy understanding of the relevant components of the coating material without a deep understanding of the chemical structure and curing chemistry, and allows for easy evaluation of the influence of each component type / amount on the color of the coating layer. In another example, the adjustment tool takes the form of a list containing the type and amount of each component, and the user can change the type / amount of a component by selecting a different component or changing the listed amount. The list is generated from the formulation data contained in the provided digital representation.In one example, touchscreen gestures are detected by the panel processor of the touchscreen panel and provided from the panel processor to the computer processor via a communication interface. Block 156 may be repeated, i.e., the formulation can be modified until the desired color is obtained.

[0205] In block 158, routine 101' generates a modified digital representation of the colored coating layer based on the user modifications detected in block 156. In this example, the detected touchscreen gestures are converted into modified formulation data of the colored coating material used to prepare the colored coating layer. Thus, the modified digital representation includes the modified coating material formulation data.

[0206] In block 160, routine 101' generates color data for the coating layer based on the modified digital representation obtained in block 158. In this example, generating color data involves providing a parameterized data-driven model to the color data of past coating layers and the past formulations of the colored coating material used to prepare the past coating layers, and calculating the color data based on the provided data-driven model and the modified formulation data of the colored coating material. The data-driven model is stored in a computer-readable medium such as a database and provided to the processor via a communication interface. An example of a suitable data-driven model is disclosed in US20020184167A1. In another example, color data is generated by retrieving color data from a database or lookup table based on the modified formulation data of the colored coating material. This example may include comparing the retrieved color data to a predefined tolerance in order to provide the best-matching color data.

[0207] In block 162, routine 101' displays the color data generated in block 160 on the display device screen. In this example, displaying the generated color data on the display device screen includes, in response to executing blocks 156 to 160, rendering the generated color data and virtual object data using the predefined illumination conditions as described above, and updating the color displayed in block 154 with the rendering result from block 162, thereby automatically updating the color of the virtual object displayed in block 154. In one example, blocks 154 to 162 are repeated. This is preferable when the displayed color does not match the color desired by the user.

[0208] In block 164, routine 101' determines whether the modified digital representation generated in block 158 is within at least one predefined tolerance, in particular all predefined tolerances. In this example, the predefined tolerances are selected from the maximum or minimum amounts of pigment and binder, the acceptable combinations of pigment and binder, and those combinations. The predefined tolerances can be determined by the coating manufacturer based on stability criteria, manufacturing process criteria, etc. The predefined tolerances are stored in a storage device such as a database and provided to the computer processor via a communication interface before determination. The determination is performed by using the computer processor to compare the data of the modified formulation of the coating material contained in the modified digital representation with at least one predefined tolerance. The result of the determination can be displayed to the user on the screen of the display device. In this example, the comparison is performed by the computer processor of the display device. In another example, the comparison is performed by a further computer processor located outside the display device. For this purpose, the modified digital representation may be provided to the further processor via a communication interface before determination, and the result of the determination may be provided to the processor of the display device via the communication interface after the determination is completed. If routine 101' determines that the modified digital representation is within at least one, preferably all, predefined tolerances, routine 101' proceeds to block 148. If routine 101' determines that the modified digital representation is outside the range of at least one predefined tolerance, routine 101' proceeds to block 168.

[0209] In block 166, routine 101' displays at least one recommendation on the display device screen if, as a result of the decision performed in block 164, the modified digital representation does not meet at least one predefined tolerance. The recommendation is stored in a database and retrieved by a processor connected to the database via a communication interface, based on the decision result, i.e., based on which tolerance is not met. In one example, the recommendation displayed is: "The amount of [compound X] is too high. Reduce the amount or select a different [compound]." The terms in parentheses are replaced in this specific example with the name of the compound that falls outside the predefined tolerance.

[0210] In block 168, routine 101' determines whether the user wishes to repeat the modification of the coating material. This may include displaying the respective menus on the GUI and prompting the user to make the appropriate selection. If routine 101 determines that the user wishes to repeat the modification of the coating material, routine 101' proceeds to block 154. Otherwise, routine 101' proceeds to block 153 as described above.

[0211] Figure 2a shows a first embodiment of a system 200 for designing the appearance of a colored coating layer manufactured from a colored coating material which can be used to carry out the method of the present invention, for example, method 100 described in relation to Figure 1. In this example, system 200 is a display device 202 coupled to databases 216, 218, 220 via communication interfaces 234, 236, 238. The display device 202 may be coupled to sensors 222, 224, 228 via communication interfaces 240, 242, 244. As shown, the computer system 200 executes instructions obtained, for example, from memory 214, and the operations related to the computer system 200, i.e., - To receive a digital representation of the colored coating layer, a digital representation of the irradiation conditions, and optionally the aforementioned model via a communication interface; - From the digital representation of the received illumination conditions, the ambient illumination conditions surrounding the display device, the orientation of the display device, and optionally the viewpoint position can be arbitrarily derived; - To generate color data for the colored coating layer based on the received digital representation of the colored coating layer, the received digital representation of the illumination conditions or the derived ambient illumination conditions surrounding the display device, the orientation of the display device, and optionally a viewpoint, and optionally a received model; - Optionally, recalculate the color data of the colored coating layer in response to changes in the digital representation of the provided illumination conditions, in particular in response to changes in data acquired by the orientation sensor and / or illumination sensor and / or viewpoint sensor; Includes a processor 212 configured to perform the following:

[0212] The display device 202 includes a screen 204 for displaying to the user, in particular via a graphical user interface (GUI), a digital representation of a provided colored coating layer, a digital representation of provided illumination conditions, and generated color data of the colored coating layer received from the processor 212. For this purpose, the display device 202 is operably coupled to the processor 212 via a communication interface. In this example, the display device 202 is connected to the processor 212 via communication interfaces 228, 230, 232, 324, a sensing circuit 208, and an input / output (I / O) controller 210. In another example, the display device 202 is directly coupled to the processor 212 via a communication interface. In this example, the display device 202 is integrated with the processor 212 and memory 214 to form a desktop computer (all-in-one machine), laptop, handheld or tablet, etc. In another example, the display device 202 may be a separate component (a peripheral device, not shown). As an example, the display device 202 may be a monochrome display, a color graphics adapter (CGA) display, an enhanced graphics adapter (EGA) display, a variable graphics array (VGA) display, a super VGA display, a liquid crystal display (e.g., active matrix, passive matrix, etc.), a cathode ray tube (CRT), a plasma display, or the like.

[0213] The system further includes an interaction element 206 for detecting user input operably coupled to the processor 212 via a communication interface. In this example, the interaction element 206 is a touchscreen operably coupled to the processor 212 via a sensing device 208 and an input / output (I / O) controller 210 so that the processor can detect user input through the interaction element 206. The touchscreen 206 is a transparent panel positioned in front of the screen 204 of the display device 202. In this example, the touchscreen 206 is integrated with the display device 202. In another example, the touchscreen 206 is a separate component. The touchscreen 206 is configured to receive input from a user touch and transmit this information to the processor 212. For this purpose, the touchscreen 206 recognizes touches on its surface, as well as the location and magnitude of the touches. The touchscreen 206 generally includes a sensing device configured to detect objects in the vicinity of it and / or pressure applied thereto, for example, by using sensing points arranged across the entire touchscreen 206. In the simplest case, a signal is generated each time an object is placed over a sensing point, for example, by using capacitance. In this example, the touchscreen 206 is connected to the touchscreen 206 via a communication interface 228 and includes a sensing circuit 208 that acquires data from the sensing device. The sensing circuit 208 is connected via a communication interface 230 to an input / output (I / O) controller 210 which is operably coupled to the processor 212. In this example, the (I / O) controller 210 is a separate component. In another example, the (I / O) controller 210 is integrated with the processor 212. The I / O controller 210 is generally configured to control instructions with one or more I / O devices, such as the touchscreen 206. The I / O controller 210 generally operates by exchanging data between the processor 212 and I / O devices that wish to communicate with the processor. Thus, the data acquired from the sensing circuit 208 is supplied to the processor 212 via the (I / O) controller 210.In another example, the processor may include this functionality. In one example, the sensing circuit 208 is configured to send raw data to the processor 212 so that the processor 212 can process the raw data. For example, the processor 212 receives the data from the sensing circuit 208 and then decides how to use the data within the system 200. The data may include the coordinates of each sensing point and the pressure applied to each sensing point. In another example, the sensing circuit 208 is configured to process the raw data itself by reading pulses from the sensing points and converting them into data that the processor 212 can understand. The sensing circuit 208 can perform filtering and / or conversion processing. Filtering is typically performed to reduce busy data streams so that the processor 212 is not overloaded with redundant or non-essential data. Conversion processing may be performed to adjust the raw data before sending or reporting it to the processor 212. Conversion may include determining the center point (e.g., the centroid) of each touch area. The sensing circuit 208 may include a memory element for storing touchscreen programs that are capable of controlling different aspects of the touchscreen 206. For example, a touchscreen program may include what type of value to output based on a selected sensing point (e.g., coordinates). The sensing circuit 208 typically includes one or more microcontrollers, each microcontroller monitoring one or more sensing points. The microcontrollers may, for example, be application-specific integrated circuits (ASICs) that work with firmware to monitor signals from the sensing device, process the monitored signals, and report this information to the processor 212. Suitable touchscreens are commercially available and are commonly found in mobile devices such as smartphones or tablets. In another example, the interaction element 206 is a mouse operably coupled to the processor 212 via a communication interface. The mouse may be connected to the processor via an input / output (I / O) controller 210, as previously mentioned in relation to the touchscreen.

[0214] Databases 216, 218, and 220 can store digital representations of colored coating layers, models derived from past illumination conditions, and / or digital representations of models derived from past environmental conditions and illumination conditions. The information stored in databases 216, 218, and 220 is retrieved by processor 212 via communication interfaces 234, 236, and 238 and used to calculate ambient illumination conditions and / or generate color data, as described in relation to step 108 in Figure 1.

[0215] The device 200 may include sensors for sensing environmental conditions, such as an orientation sensor 222, an illumination sensor 224, and / or a viewpoint sensor 226, according to various embodiments. Readings from sensors 222, 224, and 226 are provided to the processor 212 via communication interfaces 240, 242, and 244, as described in relation to step 106 of Figure 1. These readings are used by the processor 212 to calculate ambient illumination conditions using a model stored in the database 220, as described in relation to step 108 of Figure 1. The calculated ambient illumination conditions are used by the processor 212 to predict the appearance of the coated object as if the coated object were interacting with the actual environment of the device 200, as described in relation to step 108 of Figure 1. The orientation sensor 222 may sense the pitch, roll, and yaw of the device around at least one of its axes, as shown, for example, in Figures 5a and 5b. It can be understood that as the device 200 rotates, environmental factors considered when predicting the appearance of the coated object may change (e.g., illumination direction, such as the primary illumination direction, viewpoint position, etc.). Therefore, the processor 212 can take into account the pitch, roll, and yaw of the device 200 and calculate appropriate changes in environmental conditions. According to various embodiments, the resulting changes to the displayed appearance of the coated object can be updated in real time or near real time, as described in relation to step 110 in Figure 1. In this way, as the user tilts or rotates the device 200, the appearance of the coated object displayed on the screen 204 may behave as if the user were tilting or rotating an actual sample. The orientation sensor 222 can be located within the display device 202 and may comprise any suitable type of sensor capable of sensing the movement of the device 200 about one or more axes, as described above.The illumination sensor 224 can sense illumination conditions surrounding the device 200, including illumination direction such as lux level, spectral components, and primary illumination direction, or it may be used to capture at least one photograph of the environment surrounding the display device, for example, at least one HDR or LDR photograph. According to various embodiments, the illumination sensor 224 may be implemented as a CMOS imaging module, an embedded camera, or any other sensor capable of capturing images. The processor 212 can then derive ambient illumination conditions considering the captured image, as described above. The viewpoint sensor 226 is used to determine the viewpoint position and may be implemented as a CMOS imaging module, an embedded camera, or a similar device. The viewpoint position can be derived from the resulting image, as described above. Also according to various embodiments, the illumination sensor 224 and the viewpoint sensor 226 may be implemented as a single sensor 224a having both viewpoint position and illumination information derived from the image acquired by the sensor 224a, as described above.

[0216] The processor 212 may be a single-chip processor or may be implemented with multiple components. In most cases, the processor 212 works with the operating system to execute computer code and generate and use data. In this example, the computer code and data reside in memory 214 operably coupled to the processor 212. Memory 214 generally provides a place to hold data used by the computer system 200. As an example, memory 214 may include read-only memory (ROM), random access memory (RAM), a hard disk drive and / or similar. In another example, the computer code and data may reside on a removable storage medium and be loaded or installed into the computer system when needed. Removable storage media include, for example, CD-ROMs, PC-CARDs, floppy disks, magnetic tapes, and network components. In this example, the processor 212 obtains digital representations of colored coating layers and digital representations of illumination conditions from databases 216 and 218 via communication interfaces 234 and 236. For this purpose, an existing color library and a predefined HDR environment map are displayed on the screen 204 of the display device 202, and the user's selection of a color and HDR environment map is detected via the touchscreen 206, as described above. The processor 212 then retrieves a digital representation of the colored coating layer associated with the selected color from the database 216 via the communication interface 234, as described in relation to step 104 of Figure 1, and retrieves a digital representation of the illumination conditions associated with the selected HDR environment map from the database 218 via the communication interface 236, as described in relation to step 106 of Figure 1.In another example, when data is acquired from sensors 222, 224, 226 or sensors 222, 224a and provided to the processor 212 via communication interfaces 240, 242, 244, the processor 212 may further acquire from the database 220 via communication interface 238 at least one of the aforementioned models derived from past illumination conditions and / or at least one model derived from past environmental conditions. The models acquired from database 220 are used by the processor 212 to calculate ambient illumination conditions from the data acquired by sensors 222, 224, 226 or sensors 222, 224a, as described above, before generating color data, as described in relation to step 108 in Figure 1. The processor 212 then uses the acquired digital representation of the colored coating layer and the illumination conditions of the digital representation or the calculated ambient illumination conditions to generate color data by the rendering process described above, as described in relation to step 108 in Figure 1. The virtual object data used in the rendering process is stored in one of the databases 216, 218, or 220, or in a further database (not shown). The rendered object is then displayed by the display device after the processor 212 performs the rendering process by mapping the generated color data to the display device's screen, as described in relation to step 110 in Figure 1. In this example, displaying the rendered object involves updating the selected color from an existing color library. This allows the user to see the effect of selected illumination conditions on the selected color, so as to understand the effect of different illumination conditions on the selected color.

[0217] In one example, processor 212 further - To display a digital representation of the provided colored coating layer, in particular, to show at least a portion of the components of the coating material used to prepare the colored coating material related to the provided digital representation, as described in relation to step 112 in Figure 1; and - To detect user modification of the displayed digital representation of the colored coating layer, preferably by detecting user input via interaction element 206, as described in relation to step 112 in Figure 1; and - To convert detected user input into a modified digital representation of the colored coating layer; and - As described in relation to step 114 in Figure 1, generate color data for the coating layer based on the modified digital representation; and - Optionally, determine whether the generated corrected digital representation of the coating layer is within at least one predefined tolerance, and in particular within all predefined tolerances as described in relation to step 118 in Figure 1; and - Optionally, to provide a modified digital representation of the colored coating material to the coating material manufacturing site via a communication interface, as described in relation to step 120 in Figure 1; and / or - If the corrected digital representation does not meet at least one predefined tolerance, as described in relation to step 122 in Figure 1, optionally display at least one recommendation on the screen of the display device; and / or - As described in relation to step 124 in Figure 1, the order instructions for the item are provided to a further computer processor via the communication interface. It can be programmed.

[0218] Figure 2b shows a second embodiment of a system 201 for designing the color of a colored coating layer manufactured from a colored coating material which can be used to carry out the method of the present invention, for example, method 100 described in relation to Figure 1. In this example, the system 201 comprises a display device 202' which can be coupled to sensors 216', 218', 220', i.e., orientation sensor 216', illumination sensor 218', and viewpoint sensor 220', via communication interfaces 228', 230', 232'. As described in relation to Figure 2a, the illumination sensor 218' and viewpoint sensor 220' may be configured as a single sensor 218a. The display device 202' has a screen 204' for displaying to the user, in particular via a graphical user interface (GUI), a digital representation of the provided coating layer, a digital representation of the provided illumination conditions or calculated ambient illumination conditions, and generated color data received from processor 212'. For this purpose, the display device 202' is operably coupled to processor 212' via a communication interface, and processor 212' - The digital representation of the illumination conditions, i.e., the data acquired by sensors 216', 218', 220', is received via communication interfaces 228', 230', 232', as described in relation to step 108 in Figure 1; - Provide a digital representation of the received irradiation conditions to a further computing device 252' via the communication interface 244'; and - As described in relation to step 110 in Figure 1, the calculated ambient illumination conditions and generated color data are received from a further computing device 252' for display on the screen 204' of the display device 202'. It is configured in this way.

[0219] In this example, the display device 202' is connected to the processor 212' via communication interfaces 222', 224', 226', a sensing circuit 208', and an input / output (I / O) controller 210'. In another example, the display device 202' is directly coupled to the processor 212' via a communication interface. In this example, the display device 202' is integrated with the processor 212' and memory 214' to form a desktop computer (all-in-one machine), laptop, handheld or tablet, etc. In another example, the display device 202' may be a separate component (a peripheral device, not shown). In this example, the display device 202' further includes an interaction element 206' as described in relation to Figure 2a, and the processor 212' of the display device 202' is used to detect user input via the interaction element 206' as described in relation to Figures 1 and 2a.

[0220] System 201 is connected to a display device 202' via a communication interface 244' and further includes a computing device 252' connected to databases 238', 240', 242' via communication interfaces 246', 248', 250'. Databases 238', 240', 242' include digital representations of colored coating layers, digital representations of illumination conditions, and the aforementioned models derived from past illumination conditions and / or the aforementioned models derived from past environmental conditions. Computing device 234' may be a single computing device or may be located in a server environment. In the latter case, display device 202' can function as a client device and access the server (i.e., computing device 234') via a network such as the Internet (i.e., communication interface 244'). Preferably, the server may be an HTTP server and be accessed via conventional Internet web-based technology. Internet-based systems are particularly useful when a service for designing the colors of colored coating layers manufactured from colored coating materials is provided to a customer or to a large enterprise.

[0221] The computing device 252' executes instructions obtained, for example, from memory 236', and performs operations related to the system 201, i.e. - Optionally, receive a digital representation of the irradiation conditions from the processor 212' via the communication interface 244'; - Optionally, calculate the ambient illumination conditions surrounding the display device from the digital representation of the received illumination conditions and the model stored in database 242', as described in relation to step 106 in Figure 1; - Optionally providing the calculated ambient illumination conditions to the processor 212' via the communication interface 244' for display on the screen 204' of the display device 202'; and - As described in relation to step 108 in Figure 1, the color data of the colored coating layer is generated using the digital representation of the colored coating layer stored in database 238' and the digital representation of the illumination conditions stored in database 240' or the calculated ambient illumination conditions. Includes a processor 234' configured to perform the following:

[0222] By using two different computer processors to perform different steps of the method of the present invention, steps requiring high computing power can be offloaded to another computing device, and thus a display device with limited computing power can be used without adversely impacting the computation time required for generating color data, which may include using high computing power to perform calculations within an acceptable time frame. Therefore, this embodiment of the system is particularly preferred when ambient illumination conditions are calculated using the aforementioned model, or when the generation of color data is performed by rendering or computing the color data using the modified digital representation and data-driven model described above. Refer to Figure 2a for suitable computer processors 212' and 234'. In one example, the processor 334' of computing device 252' acquires a digital representation of the colored coating layer associated with the selected color, a digital representation of the illumination conditions, and virtual object data (stored in either memory 236' or a further database not shown), as described in relation to Figures 1 and 2a, and computes the color data using the rendering process described above, in particular image-based lighting. In another example, the processor 334' of computing device 252' acquires a digital representation of a colored coating layer associated with a selected color, as described in relation to Figures 1 and 2a, and calculates color data using the aforementioned calculated ambient illumination conditions and rendering process. In yet another example, the processor 334' may calculate color data based on the provided digital representation of the colored coating layer and illumination conditions such as a predefined HDR environment map in the first step, or on the provided digital representation of the colored coating layer and calculated ambient light conditions in the second step, or vice versa. The color displayed after the first step may be updated with the rendering result after performing the second step.This allows the display of the effects of different illumination conditions on the displayed colored object, enabling users to predict the appearance of the colored object using the primary illumination conditions during use, thus improving user comfort when selecting the desired color.

[0223] In one example, ambient illumination conditions calculated by processor 234' are provided to processor 212' via communication interface 244', and displayed on screen 204' of display device 202'. The calculated ambient illumination conditions may be, for example, a calculated HDR environment map. In another example, color data generated by processor 234' is provided to processor 212' via communication interface 244', and the processor maps the rendering result to screen 204' of display device 202'. In yet another example, the calculated ambient illumination conditions and generated color data are transferred to processor 212' for display on screen 204' of display device 202'. In this example, displaying the rendered object involves updating the color selected from an existing color library.

[0224] In one example, processor 212' further - To display a digital representation of the provided colored coating layer, in particular, to show at least a portion of the components of the coating material used to prepare the colored coating material related to the provided digital representation, as described in relation to step 112 in Figure 1; and - To detect user modification of the displayed digital representation of the colored coating layer, preferably by detecting user input via interaction element 206, as described in relation to step 112 in Figure 1; and - To convert detected user input into a modified digital representation of the colored coating layer; and - Optionally, determine whether the generated corrected digital representation of the coating layer is within at least one predefined tolerance, and in particular within all predefined tolerances as described in relation to step 118 in Figure 1; and - Optionally, to provide a modified digital representation of the colored coating material to the coating material manufacturing site via a communication interface, as described in relation to step 120 in Figure 1; and / or - If the corrected digital representation does not meet at least one predefined tolerance, as described in relation to step 122 in Figure 1, optionally display at least one recommendation on the screen of the display device; and / or - As described in relation to step 124 in Figure 1, the order instructions for the item are provided to a further computer processor via the communication interface. It can be programmed.

[0225] Processor 234' further - As described in relation to step 114 in Figure 1, it can be programmed to generate color data for the coating layer based on the modified digital representation.

[0226] Figure 2c shows a third embodiment of a system 201 for designing the color of a colored coating layer manufactured from a colored coating material, which can be used to carry out the method of the present invention, for example, method 100 described in relation to Figure 1. In this example, system 203 comprises a display device 202'' which can be coupled to sensors 216'', 218'', 220'', i.e., orientation sensor 216'', illumination sensor 218'' and viewpoint sensor 220'' via communication interfaces 228'', 230'', 232''. As described in relation to Figures 2a and 2b, the illumination sensor 218'' and viewpoint sensor 220'' may be configured as a single sensor 218b. The display device 202'' has a screen 204'' for displaying to the user, in particular via a graphical user interface (GUI), a digital representation of the provided coating layer, a digital representation of the provided illumination conditions or calculated ambient illumination conditions, and generated color data received from processor 212''. For this purpose, the display device 202'' is operably coupled to processor 212'' via a communication interface, and processor 212'' - The digital representation of the illumination conditions, i.e., the data acquired by sensors 216'', 218'', and 220'', is received via communication interfaces 228'', 230'', and 232'', as described in relation to step 106 in Figure 1; - As described in relation to step 106 in Figure 1, the ambient illumination conditions surrounding the display device are calculated from the digital representation of the received illumination conditions and the model stored in database 242''. It is configured in this way.

[0227] Models derived from past irradiation conditions and / or past environmental conditions, which are necessary for calculating ambient irradiation conditions from the digital representation of the received irradiation conditions, are obtained by the processor 212'' from the database 242'' via the communication interface 250''.

[0228] In this example, the display device 202'' corresponds to the display device described in Figures 2a and 2b.

[0229] System 203 further includes a computing device 252'' connected to a display device 202'' via a communication interface 244'' and to databases 238'' and 240'' via communication interfaces 246'' and 248''. Databases 238'' and 240'' include digital representations of colored coating layers and digital representations of illumination conditions. Computing device 234'' may be a single computing device or may be located in a server environment. In the latter case, the display device 202'' functions as a client device and can access the server (i.e., computing device 234'') via a network such as the Internet (i.e., communication interface 244''). Preferably, the server may be an HTTP server and may be accessed via conventional Internet web-based technology. Internet-based systems are particularly useful when a service for designing the colors of colored coating layers manufactured from colored coating materials is provided to a customer or to a large enterprise.

[0230] The computing device 252'' executes instructions obtained, for example, from memory 236'', and performs operations related to the system 203, namely, - Optionally, the ambient illumination conditions calculated by the processor 212'' may be received via the communication interface 244'; - As described in relation to step 108 in Figure 1, the color data of the colored coating layer is generated using the digital representation of the colored coating layer stored in database 238'' and the digital representation of the illumination conditions or calculated ambient illumination conditions stored in database 240''. Includes 234'' processors configured to perform the operation.

[0231] By using two different computer processors to perform different steps of the method of the present invention, steps requiring high computational power can be offloaded to another computing device, and thus a display device with limited computational power can be used without adversely impacting the computation time required for generating color data, which may include using high computational power to perform calculations within an acceptable time frame. Therefore, this embodiment of the system is particularly preferred when the generation of color data is performed by rendering or by computing the color data using the modified digital representation and data-driven model described above. Refer to Figures 2a and 2b for suitable computer processors 212'' and 234'' and for the generation of color data.

[0232] In one example, the ambient illumination conditions calculated by processor 212'' are displayed on the screen 204'' of display device 202'', as described in relation to Figure 2b. The generated color data is transferred from processor 234'' to processor 212'' via communication interface 244'' for display on the screen 204'' of display device 202'', as described in relation to Figures 2a and 2b. Displaying the rendered object may include updating the colors selected from an existing color library in the rendering result.

[0233] In one example, processor 212'' further - To display a digital representation of the provided colored coating layer, in particular, to show at least a portion of the components of the coating material used to prepare the colored coating material related to the provided digital representation, as described in relation to step 112 in Figure 1; and - To detect user modification of the displayed digital representation of the colored coating layer, preferably by detecting user input via interaction element 206, as described in relation to step 112 in Figure 1; and - To convert detected user input into a modified digital representation of the colored coating layer; and - Optionally, determine whether the generated corrected digital representation of the coating layer is within at least one predefined tolerance, and in particular within all predefined tolerances as described in relation to step 118 in Figure 1; and - Optionally, to provide a modified digital representation of the colored coating material to the coating material manufacturing site via a communication interface, as described in relation to step 120 in Figure 1; and / or - If the corrected digital representation does not meet at least one predefined tolerance, as described in relation to step 122 in Figure 1, optionally display at least one recommendation on the screen of the display device; and / or - As described in relation to step 124 in Figure 1, the order instructions for the item are provided to a further computer processor via the communication interface. It can be programmed.

[0234] The 234'' processor is further, - As described in relation to step 114 in Figure 1, generate the color data for the coating layer based on the corrected digital representation. It can be programmed.

[0235] Figure 3 shows a plan view of a system 300 comprising a display device 302 having a screen 304 on which a graphical user interface 306 showing an existing color library is displayed to the user. A suitable system is described in relation to Figures 2a-2c. The graphical user interface 304 may be shown, for example, in block 104 of Figure 1. In this example, the graphical user interface 304 is displayed on a portable display device 302, such as a smartphone. In another example, the graphical user interface 304 is displayed on a fixed display device, such as a fixed computer monitor. The graphical user interface 306 shows an existing color library on which the colors of a colored coating layer are shown on a 3D virtual object having a dome shape 308. In this example, the existing color library contains 195 colors, of which 65 are displayed on the screen of the display device. More colors can be viewed by scrolling through the existing color library 306 via touchscreen gestures. The existing color library is displayed by rendering object data of a virtual dome-shaped object and color data associated with the predefined library using predefined illumination conditions (image-based lighting) with a processor located inside the display device 302, and displaying the rendering result on the screen 304. In this example, the color data associated with the predefined library is stored in a database and provided to the display device's processor via a communication interface, preferably a wireless communication interface, before rendering, as described in relation to Figures 2a to 2c. In another example, the color data associated with the predefined color library is stored in memory located inside the display device 302. The color data associated with the colors in the pre-existing library is defined by an optimized BTF as described above.

[0236] Figure 4 shows a plan view of a system 400 comprising a display device 402 having a screen 404 on which a graphical user interface 406 is displayed to the user, showing adjustment tools 408.1, 408.2, and a virtual 3D object of a part of an automobile body 418 colored in colors related to predefined illumination conditions 416. Icons 422 indicate that the color is calculated using data on position, date, time, and optionally, empty haze. A suitable system is described in relation to Figures 2a to 2c. The graphical user interface 404 may be shown, for example, in block 106 of Figure 1.

[0237] In this example, the graphical user interface 404 includes two adjustment tools 408.1 and 408.2, which can be used to adjust the current time and haze displayed in front of the car body 418 by moving their respective regulators 410.1 and 410.2. In this example, the user can select a location by clicking the map icon 412 and navigating to the desired location using the displayed world map. In another example, the display device 404 can determine the user's current location, which is then used to calculate the ambient illumination conditions as described above. In this example, the user can select a desired date by clicking the calendar icon 414.

[0238] The graphical user interface 404 further includes a calculated HDR environment map 416 derived from predefined values ​​such as a predefined location, a predefined time, a predefined date, and a predefined empty haze, as described in relation to step 106 in Figure 1, which can be updated in real time or near real time when at least one of the location, date, time, and haze is adjusted by using the aforementioned adjustment tools 410.1, 410.2, the map icon 412, and / or the calendar icon 414.

[0239] The graphical user interface further includes a portion of the automobile body 418 colored with a color selected from an existing color library, as described, for example, in relation to step 104 in Figure 1 or Figure 3. Displaying the colored portion of the automobile body may involve rendering the color data associated with the selected color from the existing color data using a displayed HDR environment map 416 calculated from predefined position, data, time, and haze, using a model derived from past illumination conditions, i.e., I. Hosek et al., "An analytic model for full spectral sky-dome radiance," ACM Transactions on Graphics, 2012, Vol. 31, Article No.: 95, https: / / doi.org / 10.1145 / 2185520.2185591. In one example, the virtual automobile parts used for rendering are predefined virtual objects. In another example, the virtual automobile parts used for rendering can be selected by the user before rendering. The display colors of the colored car parts are updated in real time or near real time when at least one of the position, time, date, or haze is modified by recalculating the ambient illumination conditions and performing rendering using the recalculated ambient illumination conditions. The currently selected shape of the car used to display the colored car 418 on the graphical user interface 404 is indicated by a black symbol 420. The user can change the shape by clicking on the respective gray icons above the symbol 420, and the displayed colored car 418 is automatically updated in real time by performing a new rendering using the updated virtual object data. By updating the colors in real time or near real time when the illumination conditions are modified, the user can see what the colored object will look like under the selected illumination conditions, for example, under the illumination conditions that are normally present in the user's residence.This allows users to verify whether the color selected from the existing library provides the desired appearance under their specific lighting conditions, thereby improving user comfort when selecting colors for items such as automobiles and reducing user disappointment after purchase.

[0240] Figures 5a and 5b are 3D views of systems 500, 501, each comprising display devices 502, 512 with graphical user interfaces 506, 516 showing virtual 3D objects in the form of dome reliefs 508, 5018 colored with colors related to ambient illumination and the orientation of the display devices. Icons 510, 520 indicate that the colors are calculated using data obtained from orientation sensors and / or illumination sensors, and optionally viewpoint sensors. Suitable systems are described in relation to Figures 2a to 2c. The graphical user interfaces 506, 516 can be displayed, for example, in block 106 of Figure 1. The color data for the virtual dome relief 508 in Figure 5a is generated by rendering the data using ambient illumination conditions, as described in relation to step 108 of Figure 1, using color data related to colors selected from an existing library and predefined virtual object data for the dome relief, as described in relation to Figure 3. Ambient illumination conditions are calculated from data acquired from orientation sensors, illumination sensors, and optionally viewpoint sensors, as well as from at least one model derived from past environmental conditions, as described above, as implemented in, for example, Apple's ARKit or Google's ARCore program libraries.

[0241] As shown in Figure 5b, when the orientation of the display device is changed, the color data is recalculated using the changed orientation data. For this purpose, the ambient illumination conditions are recalculated using updated data from the orientation sensor, and the recalculated ambient illumination data is used for rendering. The color updates of the dome relief 518 are performed in real time or near real time so that the displayed colors behave as if the user were turning a real dome relief in their hand.

[0242] Figure 6 is a plan view of a system 600 comprising a display device 602 having a graphical user interface 606 showing available high dynamic range (HDR) environment maps 608.1, 608.n in front of a blurred shape of a car. The graphical user interface 606 may be shown, for example, in block 106 of Figure 1. The currently selected shape of the car used to display the car colored under the selected HDR environment map is shown by icon 610. Icon 612 indicates to the user that in this mode, colors will be displayed using a predefined HDR environment map. The colors displayed in this mode are selected by the user using an existing color library, as described in relation to step 104 in Figures 1 and 3. The user can select a virtual object and a desired HDR environment map by clicking on the respective icon 610 and one of the displayed predefined HDR environment maps 608.1, 608.n. The predefined HDR environment maps can be stored in a database, as described in relation to Figures 2a to 2c.

[0243] When you select an HDR environment map labeled "Virtual Object," the color data associated with the selected color from the existing color library, and the virtual object data associated with the selected virtual object, are rendered using the selected HDR environment map. The rendering result is shown in Figure 7a.

[0244] Figure 7a is a plan view of a system 700 comprising a display device 702 having a graphical user interface 706 that shows a virtual 3D object in the form of a part 708 of an automobile body colored using a selected high dynamic range (HDR) environment map. For convenience, the selected HDR environment map 714 is shown below the colored automobile. Icons 710 and 712 have the same meaning as described in relation to icons 610 and 612 in Figure 6.

[0245] The user can change the orientation of the HDR environment map 722 by clicking on a location within the displayed HDR environment map 722, and the displayed colored car can be updated in real time or near real time with the rendering results obtained when rendering color data and virtual object data using the changed orientation of the HDR environment map, as shown in Figure 7b. This allows the user to understand how the appearance of colored objects changes when different orientations of the HDR environment map are used. By using predefined HDR environment maps, the user can determine the HDR environment map that produces the desired appearance without having to install actual illumination concepts to adjust the appearance of colored objects. This can significantly reduce the costs associated with presenting colored objects, such as in a car showroom.

Claims

1. A computer implementation method for predicting the appearance of an object coated with at least one colored coating layer and displaying the predicted appearance on a display device, wherein the method is: (i) the step of providing a digital representation of the colored coating layer to a computer processor via a communication interface; (ii) - Displaying a graphical user interface containing multiple irradiation conditions on a display device; - The computer processor detects user input indicating that an irradiation condition has been selected from the displayed multiple irradiation conditions; - In response to the detected user input, the computer processor is used via the communication interface to obtain a digital representation of the irradiation conditions associated with the detected user input; The steps include providing a digital representation of the irradiation conditions; (iv) A step of generating color data for the colored coating layer using a computer processor based on a digital representation of the provided colored coating layer and a digital representation of the provided irradiation conditions; (v) The step of displaying the generated color data received from the computer processor on the display device, Methods that include...

2. The method according to claim 1, wherein the digital representation of the colored coating layer includes color space data, gloss data, appearance data, texture characteristics, or a combination thereof.

3. A method for predicting the appearance of an object coated with at least one colored coating layer and displaying the predicted appearance on a display device, the method comprising the following steps: (i) Providing a digital representation of a colored coating layer to a computer processor via a communication interface, wherein the digital representation of the colored coating layer is provided by the following steps: - A step of determining the initial BTF of the colored coating layer using a camera-based measuring device; - A step of capturing spectral reflectance data of the colored coating layer for a predetermined number, i.e., a limited number of different measurement shapes, using a spectrophotometer; - In order to obtain the optimized BTF, the initial BTF in the following equation (1) is used. [Math 1] Here, x: Surface coordinates of the sample / object [Math 2] : Irradiation and observation of the sample base coat / field of view direction [Math 3] : Color table dependent on illumination and observation direction a: Albedo or diffuse reflectance [Math 4] : The k-th Cook-Torrance lobe corresponds to the bidirectional reflectance distribution function (BRDF) that represents the gloss of the microfacet surface. S ｋ Weighting of the k-th Cook-Torrance lobe a ｋ : Parameters of the Beckmann distribution of the k-th Cook-Torrance lobe F ０，ｋ : Fresnel reflectance of the k-th Cook-Torrance lobe [Math 5] : Table of spatial texture images dependent on illumination and observation direction And, term [Math 6] and [Number 7] It is segmented into two parts, and further, the first term (F1) is defined as the irradiation and observation direction. [Number 8] The first sub-term corresponds to the color table that depends on the intensity function 【Number 9】 It is divided into a second subterm that corresponds to it, A step of fitting the initial BTF to the captured spectral reflectance data by optimizing the parameters of the first subterm in a first optimization step while keeping the parameters of the second subterm constant, and optimizing the parameters of the second subterm in a second optimization step while keeping the parameters of the first subterm constant, thereby minimizing the color difference between the captured spectral reflectance data and the initial BTF; Steps include the optimized bidirectional texture function (BTF) obtained by; (ii) - Displaying a user interface that includes multiple irradiation conditions on a display device; - The computer processor detects user input indicating that an irradiation condition has been selected from the displayed multiple irradiation conditions; - In response to the detected user input, the computer processor is used via the communication interface to obtain a digital representation of the irradiation conditions associated with the detected user input; The steps include providing a digital representation of the irradiation conditions; (iv) A step of generating color data for the colored coating layer using a computer processor based on a digital representation of the provided colored coating layer and a digital representation of the provided irradiation conditions; (v) The step of displaying the generated color data received from the computer processor on the display device, Methods that include...

4. The method according to any one of claims 1 to 3, wherein providing a digital representation of the colored coating layer includes displaying an existing color library on the screen of the display device, selecting a color from the displayed existing library, obtaining a digital representation of the colored coating layer based on the selected color, and providing the obtained digital representation of the colored coating layer to the computer processor via the communication interface.

5. To obtain a digital representation of the aforementioned irradiation conditions, - To obtain data indicating the date, time, location, and in particular geographic location via the aforementioned communication interface, and / or - To acquire data acquired from at least one irradiation sensor of the display device and data acquired from at least one orientation sensor of the display device via the communication interface, and / or - To acquire at least one high dynamic range (HDR) environment map via the aforementioned communication interface, The method according to claim 1 or 3, including the method described in claim 1 or 3.

6. The method according to claim 5, comprising: obtaining data indicating the date and / or time and / or degree of empty haze via the communication interface; displaying at least one tuning tool including at least one regulator corresponding to the date and / or time or haze; detecting user input via an interaction element indicating operation of the at least one tuning tool, in particular by detecting the movement of at least one regulator of the at least one tuning tool via the interaction element; and determining the date and / or time or haze in response to the detected user input.

7. The method according to claim 1 or 3, wherein the data acquired from at least one irradiation sensor of the display device includes data relating to the irradiation conditions surrounding the display device, such as lux level, spectral components, irradiation direction, at least one photograph of the environment surrounding the display device, in particular at least one high dynamic range (HDR) or low dynamic range (LDR) photograph, or a combination thereof.

8. The method according to claim 1 or 3, wherein the model derived from the aforementioned past irradiation conditions is a physically based analytical model of the daytime sky.

9. The method according to claim 1 or 3, wherein at least one model derived from the aforementioned past environmental conditions provides a relationship between data acquired from the illumination sensor and / or orientation sensor and / or viewpoint sensor of the display device and the ambient light conditions surrounding the display device.

10. The method according to claim 1 or 3, wherein the step of generating color data in the computer processor includes providing object data of a virtual object, mapping a digital representation of the provided colored coating layer onto the provided virtual object, and rendering the result of the mapping using a digital representation of the provided illumination conditions.

11. The method according to claim 1 or 3, further comprising, step (iv), calculating the ambient illumination conditions surrounding the display device using the computer processor from a digital representation of the provided illumination conditions and at least one model derived from the provided historical environmental conditions, before generating the color data for the colored coating layer.

12. The method according to claim 1 or 3, wherein the step of displaying the color data received from the computer processor on the screen of the display device includes mapping the generated color data, in particular each rendered point, to the screen of the display device.

13. A system for predicting the appearance of an object coated with at least one colored coating layer, wherein the system: - At least one communication interface for providing the computer processor with a digital representation of the colored coating layer and a digital representation of the irradiation conditions; - A display device equipped with a screen; - Interaction elements for detecting user input; - The communication interface, a processor that communicates with the display device, wherein the processor is: - Receive the digital representation of the colored coating layer via the communication interface; - Generate a user interface presentation that includes multiple irradiation conditions, detect user input indicating that an irradiation condition has been selected from the displayed multiple irradiation conditions, and in response to the detected user input, obtain a digital representation of the irradiation condition associated with the detected user input; - Based on the received digital representation of the colored coating layer and the received digital representation of the illumination conditions or the calculated ambient illumination conditions surrounding the display device, color data of the colored coating layer is generated. The processor is programmed to do so, Equipped with, The display device receives the generated user interface presentation and the generated color data of the colored coating layer from the processor, and displays the generated user interface presentation and color data. The digital representation of the aforementioned colored coating layer is performed in the following steps: - A step of determining the initial BTF of the colored coating layer using a camera-based measuring device; - A step of capturing spectral reflectance data of the colored coating layer for a predetermined number, i.e., a limited number of different measurement shapes, using a spectrophotometer; - In order to obtain the optimized BTF, the initial BTF in the following equation (1) is used. [Number 10] Here, x: Surface coordinates of the sample / object [Math 11] : Irradiation and observation of the sample base coat / field of view direction [Math 12] : Color table dependent on illumination and observation direction a: Albedo or diffuse reflectance [Number 13] : The k-th Cook-Torrance lobe corresponds to the bidirectional reflectance distribution function (BRDF) that represents the gloss of the microfacet surface. S ｋ Weighting of the k-th Cook-Torrance lobe a ｋ : Parameters of the Beckmann distribution of the k-th Cook-Torrance lobe F ０，ｋ : Fresnel reflectance of the k-th Cook-Torrance lobe [Number 14] : Table of spatial texture images dependent on illumination and observation direction And, term [Number 15] and [Number 16] It is segmented into two parts, and further, the first term (F1) is defined as the irradiation and observation direction. [Number 17] The first sub-term corresponds to the color table that depends on the intensity function [Number 18] It is divided into a second subterm that corresponds to it, A step of fitting the initial BTF to the captured spectral reflectance data by optimizing the parameters of the first subterm in a first optimization step while keeping the parameters of the second subterm constant, and optimizing the parameters of the second subterm in a second optimization step while keeping the parameters of the first subterm constant, thereby minimizing the color difference between the captured spectral reflectance data and the initial BTF; A system including an optimized bidirectional texture function (BTF) obtained by [the specified method].

14. A non-transient computer-readable storage medium, wherein the computer-readable storage medium includes an instruction that, when executed by a computer, causes the computer to perform a step according to the method of claim 1 or 3.

15. The use of the method according to claim 1 or 3 for predicting the appearance of an object coated with at least one colored coating layer.

16. The aforementioned method, (i) the step of providing a digital representation of the colored coating layer to a computer processor via a communication interface; (ii) - Displaying a graphical user interface containing multiple irradiation conditions on a display device; - The computer processor detects user input indicating that an irradiation condition has been selected from the displayed multiple irradiation conditions; - In response to the detected user input, the computer processor is used via the communication interface to obtain a digital representation of the irradiation conditions associated with the detected user input; The steps include providing a digital representation of the irradiation conditions; (iii) Providing the computer processor via the communication interface a model derived from past irradiation conditions and / or at least one model derived from past environmental conditions; (iv) The steps of generating color data for the colored coating layer using a computer processor based on the provided digital representation of the colored coating layer, the provided digital representation of the irradiation conditions, and the provided model; (v) The step of displaying the generated color data received from the computer processor on the display device, The method according to claim 1, including the method described in claim 1.

17. The above method involves the following steps: (i) Providing a digital representation of a colored coating layer to a computer processor via a communication interface, wherein the digital representation of the colored coating layer is provided by the following steps: - A step of determining the initial BTF of the colored coating layer using a camera-based measuring device; - A step of capturing spectral reflectance data of the colored coating layer for a predetermined number, i.e., a limited number of different measurement shapes, using a spectrophotometer; - In order to obtain the optimized BTF, the initial BTF in the following equation (1) is used. [Number 19] Here, x: Surface coordinates of the sample / object [Number 20] : Irradiation and observation of the sample base coat / field of view direction [Math 21] : Color table dependent on illumination and observation direction a: Albedo or diffuse reflectance [Number 22] : The k-th Cook-Torrance lobe corresponds to the bidirectional reflectance distribution function (BRDF) that represents the gloss of the microfacet surface. S ｋ Weighting of the k-th Cook-Torrance lobe a ｋ : Parameters of the Beckmann distribution of the k-th Cook-Torrance lobe F ０，ｋ : Fresnel reflectance of the k-th Cook-Torrance lobe [Number 23] : Table of spatial texture images dependent on illumination and observation direction And, term [Number 24] and [Number 25] It is segmented into two parts, and further, the first term (F1) is defined as the irradiation and observation direction. [Number 26] The first sub-term corresponds to the color table that depends on the intensity function [Number 27] It is divided into a second subterm that corresponds to it, A step of fitting the initial BTF to the captured spectral reflectance data by optimizing the parameters of the first subterm in a first optimization step while keeping the parameters of the second subterm constant, and optimizing the parameters of the second subterm in a second optimization step while keeping the parameters of the first subterm constant, thereby minimizing the color difference between the captured spectral reflectance data and the initial BTF; Steps include the optimized bidirectional texture function (BTF) obtained by; (ii) - Displaying a user interface that includes multiple irradiation conditions on a display device; - The computer processor detects user input indicating that an irradiation condition has been selected from the displayed multiple irradiation conditions; - In response to the detected user input, the computer processor is used via the communication interface to obtain a digital representation of the irradiation conditions associated with the detected user input; The steps include providing a digital representation of the irradiation conditions; (iii) Providing the computer processor via the communication interface a model derived from past irradiation conditions and / or at least one model derived from past environmental conditions; (iv) The steps of generating color data for the colored coating layer using a computer processor based on the provided digital representation of the colored coating layer, the provided digital representation of the irradiation conditions, and the provided model; (v) The step of displaying the generated color data received from the computer processor on the display device, The method according to claim 3, including the method described in claim 3.

18. To obtain a digital representation of the aforementioned irradiation conditions, - To obtain data indicating the date, time, location, in particular geographic location, and the degree of haze in the sky via the aforementioned communication interface, and / or - To acquire data acquired from at least one illumination sensor of the display device, data acquired from at least one orientation sensor of the display device, and / or data acquired from at least one viewpoint sensor via the communication interface, and / or - To acquire at least one high dynamic range (HDR) environment map via the aforementioned communication interface, The method according to claim 5, including the method described in claim 5.

19. The method according to claim 10, wherein the step of generating color data with the computer processor includes providing object data of a virtual object, providing further color data for at least one further coating layer, mapping the digital representation of the provided colored coating layer and the further color data onto the provided virtual object, and rendering the result of the mapping using the digital representation of the provided illumination conditions and the provided model.

20. The aforementioned system: - A communication interface for providing a computer processor with a model derived from past irradiation conditions and / or at least one model derived from past environmental conditions; - At least one communication interface for providing the computer processor with a digital representation of the colored coating layer and a digital representation of the irradiation conditions; - A display device equipped with a screen; - Interaction elements for detecting user input; - At least one illumination sensor and / or at least one orientation sensor adapted to sense the orientation of the display device and / or at least one viewpoint sensor adapted to sense the viewpoint of a user holding the display device; - A processor that communicates with the communication interface, the display device, and the at least one illumination sensor and / or orientation sensor and / or viewpoint sensor, wherein the processor: - Receive the digital representation of the colored coating layer via the communication interface; - Generate a user interface presentation that includes multiple irradiation conditions, detect user input indicating that an irradiation condition has been selected from the displayed multiple irradiation conditions, and in response to the detected user input, obtain a digital representation of the irradiation condition associated with the detected user input; - From the digital representation of the received irradiation conditions and the received model, the ambient irradiation conditions surrounding the display device are calculated; - Based on the received digital representation of the colored coating layer and the received digital representation of the illumination conditions or the calculated ambient illumination conditions surrounding the display device, color data of the colored coating layer is generated. The processor is programmed to do so, Equipped with, The display device receives the generated user interface presentation and the generated color data of the colored coating layer from the processor, and displays the generated user interface presentation and color data. The digital representation of the aforementioned colored coating layer is performed in the following steps: - A step of determining the initial BTF of the colored coating layer using a camera-based measuring device; - A step of capturing spectral reflectance data of the colored coating layer for a predetermined number, i.e., a limited number of different measurement shapes, using a spectrophotometer; - In order to obtain the optimized BTF, the initial BTF in the following equation (1) is used. [Number 28] Here, x: Surface coordinates of the sample / object [Number 29] : Irradiation and observation of the sample base coat / field of view direction [Number 30] : Color table dependent on illumination and observation direction a: Albedo or diffuse reflectance [Number 31] : The k-th Cook-Torrance lobe corresponds to the bidirectional reflectance distribution function (BRDF) that represents the gloss of the microfacet surface. S ｋ Weighting of the k-th Cook-Torrance lobe a ｋ : Parameters of the Beckmann distribution of the k-th Cook-Torrance lobe F ０，ｋ : Fresnel reflectance of the k-th Cook-Torrance lobe [Number 32] : Table of spatial texture images dependent on illumination and observation direction And, term [Number 33] and [Number 34] It is segmented into two parts, and further, the first term (F1) is defined as the irradiation and observation direction. [Number 35] The first sub-term corresponds to the color table that depends on the intensity function [Number 36] It is divided into a second subterm that corresponds to it, A step of fitting the initial BTF to the captured spectral reflectance data by optimizing the parameters of the first subterm in a first optimization step while keeping the parameters of the second subterm constant, and optimizing the parameters of the second subterm in a second optimization step while keeping the parameters of the first subterm constant, thereby minimizing the color difference between the captured spectral reflectance data and the initial BTF; The system according to claim 13, comprising an optimized bidirectional texture function (BTF) obtained by the method.

Images