Visualizing the appearance of at least two materials

Abstract

A first and second set of appearance attributes are obtained. The first set is associated with the target material. It includes measured appearance attributes of the target material (54). The second set is associated with the candidate material. It includes candidate appearance attributes based on appearance attributes associated with one or more reference materials. A geometric model of at least one virtual object (72) is obtained, the geometric model defining a three-dimensional surface geometry. A scene including the at least one virtual object (72) is visualized. First and second portions of the virtual object (72) are visualized using the first and second sets of appearance attributes, respectively. Each of the first and second sets of appearance attributes includes texture attributes in the form of a plurality of image datasets. The image data in one of the sets can be based on the texture attributes of the other set, or the image data in the second set can be synthesized from image data associated with a plurality of component materials of the candidate material.

Description

Technical Field

[0001] This invention relates to a method for visualizing the appearance of at least two materials. The invention also relates to apparatus for performing such a method and a corresponding computer program. Background Technology

[0002] Finding a color formulation that matches a given color to a physical object (“target object”) can be a lengthy trial-and-error process. A set of colorants is selected, candidate formulations using those colorants are searched or formulated, and test objects, such as test panels, are prepared by applying materials made according to the candidate formulations to the test objects. The test objects are then visually compared to the target object. If the color of the test object does not match the target object, the candidate formulation is repeatedly modified as needed until the color of the test object matches the target object within the desired tolerance range.

[0003] In some applications, the target object may be a vehicle component coated with an existing coating, and the goal is to find a coating formulation whose color matches the appearance of the existing coating. Even if the paint code or vehicle identification number associated with the target object is known, simply retrieving the corresponding reference formulation may not yield an acceptable match. This is because, even with paint code specifications and reference formulations, the color of a given target object will vary slightly depending on the batch, formulator, or year. Modifications to the reference formulation need to be considered.

[0004] Traditionally, the process of finding a matching formulation has been entirely manual. Whether formulating from scratch or modifying a reference formulation, success largely depends on the experience of the color professionals who select colorants and define candidate formulations. Even experienced experts often need several iterations until a satisfactory match is achieved.

[0005] Over the past few decades, increasingly sophisticated color matching software has been introduced to assist color professionals in achieving color matching within defined tolerance ranges with fewer iterations. Examples include "Color iMatch," software available from X-Rite, a company based in Grand Rapids, Michigan. TM The software available is PaintManager, from PPG Industries, Strongsville, Ohio; Match Pigment, from Datacolor, Lawrenceville, New Jersey; or Colibri, from Konica Minolta, Osaka, Japan. TMColorMatch. Color matching software typically has three main components (see, for example, [Ber19]): the first is a database of the optical properties of colorants. The second is a set of algorithms for selecting colorants and predicting candidate formulations. The third is a set of algorithms for correcting the initial candidate formulations when the match is not within tolerance.

[0006] To populate a database with the optical properties of colorants, these properties need to be determined through measurement. This can be done in different ways depending on the type of application. For example, for coatings commonly used in the automotive industry, mixtures of each colorant can be prepared with a base at different concentrations, and possibly with varying amounts of added white and black pigments. Reference objects can be created for each mixture in a so-called drawdown form (e.g., as a black and white opaque card coated with that mixture). The drawdown color properties, in the form of spectral data or in a predefined color space (e.g., trichromatic values ​​or CIELAB values), can then be determined using instruments such as a spectrophotometer and fed into the database.

[0007] To perform color matching on a physical target object, the color properties of the target object are also determined. The formulation software then predicts one or more candidate formulations, which are expected to produce color properties approximately identical to the target object. To evaluate the quality of the match, test objects (e.g., reductions in the case of coatings) can be created for the selected candidate formulations. The test objects can then be visually compared to the target object. If the match is still not visually satisfactory, the color properties of the test objects can be acquired, and the color formulation software can correct the formulation based on a comparison between the calculated color properties of the candidate formulations and the measured color properties of the test objects. This process can be repeated frequently as needed until an acceptable match is achieved.

[0008] While acceptable results can often be obtained with fewer iterations compared to traditional manual methods, there is still room for improvement. In particular, it is desirable to assess the quality of the match between the candidate formulation and the target material with increased confidence before the test subject has been actually produced. This is especially desirable if the location where the test subject is produced differs from the location where the target material is located. For example, the target material could be part of a damaged vehicle that needs repair in a body shop. However, the body shop itself may not have color mixing facilities. Instead, the body shop may have to order paint ready for application from a remote paint supplier. In such cases, a match within tolerance on the first shot may be necessary.

[0009] Known color matching methods may not yield satisfactory results, especially for painted coatings or other materials exhibiting gonioapparent properties. For example, some materials exhibit color flops. A color flop is a change in the color value, hue, or chromaticity of a material's appearance as the direction of illumination and observation changes. Other examples of materials with gonioapparent properties include those containing effect pigments, such as metallic flakes that produce a shimmering effect or interference flakes that produce a pearlescent effect, as well as materials with non-planar surface microstructures. In such cases, simply matching the color may be insufficient. Instead, the entire appearance needs to be matched, including angle-dependent color and texture. Known techniques generally fail to provide satisfactory visualizations or measurements to accurately match the appearance of such materials.

[0010] US20150026298A1 discloses a method for selecting the most probable variant of a candidate paint color standard for vehicle repair using a mobile device. In this method, a user at a body shop inputs information about the vehicle's color into a mobile device, which transmits this information to a remote central computer. The central computer selects a candidate color standard and transmits information about that candidate color standard back to the mobile device. The mobile device displays information about the selected candidate color standard. The user then visually compares a physical image representing the selected candidate color standard with the color of the vehicle to be repaired. A drawback of this method is the need for a large number of physical images of the color standard at the body shop. Alternatively, an image of the selected candidate color standard can be displayed on a color display on the mobile device, and the user visually compares the displayed image with the color of the vehicle to be repaired. However, this requires careful calibration of the display to enable a meaningful comparison.

[0011] US20070097119A1 discloses a method for displaying a simulated paint coating on a display device. RGB color values ​​are calculated over a range of non-specular angles. A statistical texture function for the paint coating is determined. The statistical texture function is applied to the RGB values, and these values ​​are used to display the color pixels. The statistical texture function is independent of the illumination and viewing direction.

[0012] US20050128484A1 discloses a method for determining a repair paint formula for color matching. The color characteristics of the target color to be matched are identified, input, and processed in such a way that the target color can be visually displayed. Alternative colors are selected from a database. The alternative colors can be displayed on a monitor as several virtual sheets, each representing a different viewing angle, or displayed as a curved panel. An image showing the appearance characteristics of the sheet can be overlaid with the color. The virtual sheets can be viewed in conjunction with the target color. Interpolation can be performed on the colors and images obtained for multiple non-mirror angles to show the change in the sheet appearance with varying non-mirror angles. This document does not describe how the color and sheet appearance are interpolated.

[0013] Both US20080291449A1 and US20080235224A1 disclose methods for displaying images to select matching formulations to match the appearance of an article, such as a target coating on a vehicle. In one embodiment, color data of the article is obtained using a colorimeter or spectrophotometer. Texture data of the article is obtained using an imaging device. A target image is created based on the color and texture data and displayed on a display device. A preliminary matching formulation is manually retrieved from a database. A matching image is generated for the preliminary matching formulation. The matching image and the target image are displayed side-by-side on the display device. The target image and the matching image can be displayed for multiple non-mirror angles. The aforementioned documents do not mention how to generate and display texture data for multiple non-mirror angles.

[0014] US20200089991A1 discloses a system for displaying one or more images to select one or more matching formulations to match the appearance of a target coating on an article. A first database contains repair formulations and associated appearance characteristics. A second database contains identification information or three-dimensional geometric data of at least one article. A preliminary matching formulation is retrieved from the first database, an article or its three-dimensional geometric data is selected from the second database, and marked portions of the article's surface are received. An individual matching image containing the marked portions and unmarked portions adjacent to the marked portions is generated and displayed on a display device. In the marked portions, the individual matching image is generated based on the appearance characteristics of the preliminary matching formulation. In the unmarked portions, the individual matching image is generated based on the appearance characteristics of the article. Appearance characteristics are calculated from images acquired from the article using an imaging device. Appearance characteristics may include texture, metallic or pearlescent effects, gloss, image sharpness, flake appearance (such as texture, glitter, reflection, and shimmer), and enhancement of depth perception imparted by the flakes. This document does not specify how the appearance of the article is displayed for different illumination and viewing directions.

[0015] US8,872,811B1 discloses a method for digitally generating data that indicates the synthetic appearance of a simulated material having physically plausible appearance properties. The dataset indicating the synthetic appearance of the simulated material is determined partly based on data associated with a physically tangible source material and partly based on data of measured properties of a physically tangible reference material.

[0016] US2007291993A1 discloses an apparatus for measuring the spatial undersampled bidirectional reflection distribution function (BRDF) of a surface. Summary of the Invention

[0017] The object of this invention is to provide a method for visualizing the appearance of target materials and candidate materials included in a target object, wherein the formulation of the candidate materials has been determined by formulation software in such a way that the user can determine with increased confidence whether the appearance of the materials matches, without the need to produce physical test objects.

[0018] This invention provides a computer-implemented method for visualizing the appearance of at least two materials, the method comprising:

[0019] Obtain a first set of appearance attributes, the appearance attributes in the first set being associated with the target material, the first set including the measured appearance attributes of the target material, the measured appearance attributes having been determined by performing measurements on a target object including the target material;

[0020] Obtain the second set of appearance attributes, which are associated with the candidate materials;

[0021] Obtain a geometric model of at least one virtual object, the geometric model defining the three-dimensional macroscopic surface geometry of the at least one virtual object; and

[0022] The scene including the at least one virtual object is visualized using a display device, first and second sets of appearance attributes, and the geometric model; a first portion of the at least one virtual object is visualized using the first set of appearance attributes; and a second portion of the at least one virtual object is visualized using the second set of appearance attributes.

[0023] Each of the first and second groups of appearance attributes includes a texture attribute in the form of image data, wherein the image data in the first group is calculated based on texture attributes associated with one or more reference materials and / or with candidate materials, or the image data in the second group is calculated based on texture attributes associated with target materials and / or with test subjects including candidate materials.

[0024] Specifically, the measured appearance properties of the target material may lack texture properties, or may contain only a limited set of texture properties. This may occur, for example, if the appearance capture device used to measure the target object lacks imaging capability or has only very limited imaging capability. In this case, the measured appearance properties of the target material can be supplemented by texture properties from different sources. Specifically, the first set of appearance properties (i.e., the set associated with the target material) may include texture properties, particularly texture properties in the form of image data, which have been determined based on texture properties associated with one or more reference materials and / or with candidate materials. Specifically, some or all of the texture properties in the first set may be determined based on calculated texture properties associated with candidate materials (particularly in the form of image data) or based on measured texture properties associated with test objects including candidate materials (particularly in the form of image data). Therefore, the method may include determining texture properties (particularly in the form of image data) in the first set of appearance properties based on texture properties (particularly in the form of image data) associated with one or more reference materials and / or with candidate materials. This could involve modifying pixel values ​​in image data associated with reference materials and / or candidate materials to match at least one statistical property of the first set with known statistical properties of the target material. In this way, the target material can be visualized as having a realistic texture, even if the measured texture information itself is insufficient to achieve this purpose.

[0025] In other embodiments, the available appearance attributes of candidate or reference materials may lack texture attributes, or may contain only a limited set of texture attributes. This may occur, for example, if the database containing the appearance attributes of the reference materials was created using a simpler or older appearance capture device with limited or no imaging capabilities. In this case, the appearance attributes in the second set (i.e., the set associated with the candidate materials) may be supplemented by texture attributes (particularly in the form of image data) from different sources. Specifically, the second set of appearance attributes may include texture attributes (particularly in the form of image data) already determined based on measured texture attributes (particularly in the form of image data) associated with the target material and / or with test subjects including the candidate materials. In particular, some or all of the texture attributes in the second set may be determined based on measured texture attributes (particularly in the form of image data) associated with the target material or based on measured texture attributes (particularly in the form of image data) associated with test subjects including the candidate materials. Therefore, the method may include determining texture attributes in the second set of appearance attributes based on measured texture attributes associated with the target material and / or based on texture attributes associated with test subjects including the candidate materials. This could involve modifying pixel values ​​in image data associated with the target material and / or the test subject to match at least one statistical property of the second set with known statistical properties of the candidate material. In this way, the candidate material can be visualized as having a realistic texture, even if the available texture information itself is insufficient to achieve this purpose.

[0026] In another aspect, the present invention provides a computer-implemented method for visualizing the appearance of at least two materials, the method comprising:

[0027] Obtain a first set of appearance attributes, the appearance attributes in the first set being associated with the target material, the first set including the measured appearance attributes of the target material, the measured appearance attributes having been determined by performing measurements on a target object including the target material;

[0028] Obtain the second set of appearance attributes, which are associated with the candidate materials;

[0029] Obtain a geometric model of at least one virtual object, the geometric model defining the three-dimensional macroscopic surface geometry of the at least one virtual object; and

[0030] The scene including the at least one virtual object is visualized using a display device, first and second sets of appearance attributes, and the geometric model; a first portion of the at least one virtual object is visualized using the first set of appearance attributes; and a second portion of the at least one virtual object is visualized using the second set of appearance attributes.

[0031] Each of the first and second groups of appearance attributes includes a texture attribute in the form of image data.

[0032] The candidate material is a composite material comprising at least two components according to a formulation that defines the concentration of each component in the composite material.

[0033] The second set of appearance attributes includes at least one target texture, which indicates the spatial variation of the composite material's appearance, and

[0034] The method includes generating a target texture using multiple source textures, each source texture being associated with one of the components.

[0035] Generating the target texture can include:

[0036] (i) Assign interpolation weights to each of the source textures; and

[0037] (ii) Synthesize the target texture using the source texture and the assigned interpolation weights.

[0038] The target texture for synthesis can include:

[0039] (a) Randomly select one of the source textures with a probability proportional to the interpolation weight of the source texture;

[0040] (b) Randomly extract texture patches from the selected source texture;

[0041] (c) Modify the extracted texture by modifying the pixel values ​​in the extracted texture to obtain a modified texture, wherein the pixel value modification is performed in such a way that at least one statistical property of the modified texture approximates the corresponding average statistical property, and the average statistical property is determined by performing a weighted average on the source texture, which is weighted by interpolation weights.

[0042] (d) Insert the modified texture patch into the target texture so that the modified texture patch seamlessly fits the existing texture content in the target texture; and

[0043] (e) Repeat steps (a)-(d) until the target texture is completely filled.

[0044] The method may further include adjusting pixel values ​​in the target texture to correct for absorption and scattering effects in the composite material. Pixel values ​​can be adjusted such that at least one statistical property of the adjusted target texture matches a reference property of the composite material. Specifically, pixel values ​​can be adjusted such that the average color space value of the adjusted target texture matches a reference color space value of the composite material, wherein the color space values ​​and the reference color space value are preferably expressed in a perceptual color space.

[0045] If the target texture indicates the spatial variation of the composite material’s appearance for a specific combination of illumination and viewing directions, and if a reference color space value is not available for that specific combination, the method may include fitting the parameters of a BRDF model to available reference colors for a plurality of other combinations of illumination and viewing directions, and evaluating the BRDF model for said specific combination to obtain a reference color space value for that specific combination.

[0046] In an alternative embodiment, to correct the absorption and scattering effects in the composite material, the method may include:

[0047] To obtain individual optical parameters that at least approximately describe the scattering and absorption behavior of each component in the composite material;

[0048] The combined optical parameters describing the scattering and absorption behavior of composite materials are determined based on the concentration of the components and their individual optical parameters.

[0049] Optical simulations of the luminous flux within the composite material are performed on at least one layer beneath the surface of the composite material to determine the attenuation factors of incident and reflected light for the effect pigments in the layer; and

[0050] The pixel values ​​of the target texture are adjusted based on the attenuation factor.

[0051] The step of obtaining a second set of appearance attributes associated with a candidate material is not limited to a single candidate material. For example, additional sets of appearance attributes can also be obtained, each associated with a different candidate material, and these can be combined with other steps for visualization of more than one candidate material.

[0052] In this invention, target and candidate materials are visualized in the form of at least one virtual object. The geometry of the virtual object is described by a three-dimensional geometric model. This geometric model defines the three-dimensional macroscopic surface geometry of the virtual object. Preferably, the macroscopic surface geometry has at least one portion that is curved along at least two mutually orthogonal directions. In some embodiments, the curved portion of the virtual object can be represented by a combination of small polygonal planar surfaces (e.g., a polygonal mesh) in the geometric model. Preferably, the curved three-dimensional macroscopic surface geometry includes both convex and concave portions. Preferably, the virtual object has a surface normal that covers a large solid angle, i.e., has a wide range of directions in three-dimensional space. Preferably, the solid angle covered by the direction of the surface normal of the virtual object is at least 50% of the solid angle of a hemisphere, i.e., it is preferably at least 1πsr. In this way, effects such as color shift, gloss, glitter, and texture can be compared between the two materials simultaneously in a large number of illumination and observation directions relative to the surface normal of the virtual object.

[0053] A visualized scene may include more than one virtual object. For example, a scene may include two identically shaped virtual objects visualized side-by-side in the same direction, one visualized using a first set of appearance attributes and the other visualized using a second set of appearance attributes. In other embodiments, a scene may include a single virtual object having a first portion visualized using a first set of attributes and a second portion visualized using a second set of attributes.

[0054] As used in this disclosure, the term “appearance” includes both color and texture, with “texture” being broadly understood as describing the spatial variation in appearance across the surface of an object.

[0055] In a preferred embodiment, the at least one virtual object is visualized (i.e., rendered and displayed) using texture properties comprising or derived from multiple image datasets, each image dataset being associated with a different combination (e.g., different pairs) of illumination and viewing directions. By using image data for multiple combinations of illumination and viewing directions, particularly realistic visualizations of the target and candidate materials are obtained, and the user can determine with high confidence whether the appearances of these materials match.

[0056] Texture can be highly dependent on the direction of illumination and viewing. For example, in metallic effect paints that typically contain highly reflective flakes, the location of the surface where strong reflections are observed may vary discontinuously when the direction of illumination and / or viewing changes continuously, because the flakes at different locations across the surface will reflect under different combinations of illumination and viewing directions. Therefore, the observed texture can be very different between illumination and viewing directions that differ by only small angular values. Thus, using texture properties in the form of image data is particularly useful for matching materials containing reflective flakes, regardless of the different combinations of illumination and viewing directions.

[0057] The first set of appearance attributes associated with the target material includes the measured appearance attributes of the target material. The proposed method may include the step of actually performing measurements on a target object including the target material using an appearance capture device.

[0058] A second set of appearance attributes associated with the candidate material includes candidate appearance attributes. In some embodiments, the candidate material is a known reference material, and therefore the candidate appearance attributes are appearance attributes associated with the known reference material. In other embodiments, the candidate material is composed of multiple component materials, and the appearance attributes can be used for multiple reference materials including the component materials of the candidate material. Therefore, by using calculations, candidate appearance attributes can be based on predetermined appearance attributes associated with multiple reference materials. A formulation can be determined in such a way that the candidate material has expected appearance attributes that at least approximately match the appearance attributes of the target material. In particular, the proposed method may include the step of determining a formulation. This may involve minimizing a measure of difference between the expected appearance attributes of the candidate material and the measured appearance attributes of the target material. Suitable measures of difference and optimization algorithms are well known in the field of color formulation.

[0059] In some embodiments, the method further includes, for example, obtaining a physical, tangible test object comprising candidate materials by producing test objects. The method may include determining measured appearance properties of the test object by performing measurements on the test object using an appearance capture device. The method may also include visualizing at least a portion of the at least one virtual object using the measured appearance properties of the test object. In this way, the appearance of the test object can be compared to the appearance of the target object even if the target object does not exist in the same location as the test object. Additionally or alternatively, the method may include determining a modified formulation using the measured appearance properties of the test object and the calculated appearance properties of the candidate materials. Once the measured appearance properties of the test object are available, suitable algorithms for determining the modified formulation are well known in the field of color formulation. Such algorithms are often referred to as “correction algorithms.” The method can then be repeated with the modified formulation.

[0060] The appearance capturing device used to determine the measured appearance attributes of the test subject can be the same as the device used to determine the measured appearance attributes of the target object, or it can be a different device. The device is preferably configured to determine the same type of appearance attribute under the same illumination and observation conditions as the appearance capturing device used for the target object. However, it is also conceivable that the appearance capturing device for the test subject can be configured to determine appearance attributes of different types, different numbers of appearance attributes, and / or different illumination and observation directions than the appearance capturing device used for the target object.

[0061] In each case, the appearance capturing device can be a multi-angle spectrophotometer configured to determine color attributes for multiple combinations of illumination and observation directions. In addition to color attributes, the appearance capturing device can also have imaging capabilities to determine texture attributes.

[0062] In some embodiments, the measured appearance properties of the target material include multiple measured image datasets, each associated with a different combination (e.g., different pairs) of illumination and viewing directions. For this purpose, an appearance capture device for determining the appearance properties of the target material may include one or more cameras configured to determine image data for multiple combinations (e.g., multiple pairs) of illumination and viewing directions. In some embodiments, all combinations of illumination and viewing directions of the measured image data include the same viewing direction but multiple different illumination directions (or equivalently, the same illumination direction and multiple different viewing directions). This may be the case, for example, when using an appearance capture device with a single camera and multiple different light sources. In other embodiments, the combination of illumination and viewing directions includes two or more viewing directions.

[0063] In another aspect, the present invention provides a computer-implemented method for visualizing the appearance of at least two materials, the method comprising:

[0064] Obtain a first set of appearance attributes, the appearance attributes in the first set being associated with the target material, the first set including the measured appearance attributes of the target material, the measured appearance attributes having been determined by performing measurements on a target object including the target material;

[0065] Obtain the second set of appearance attributes, which are associated with the candidate materials;

[0066] Obtain a geometric model of at least one virtual object, the geometric model defining the three-dimensional macroscopic surface geometry of the at least one virtual object; and

[0067] The scene including the at least one virtual object is visualized using a display device, first and second sets of appearance attributes, and the geometric model; a first portion of the at least one virtual object is visualized using the first set of appearance attributes; and a second portion of the at least one virtual object is visualized using the second set of appearance attributes.

[0068] Each of the first and second groups of appearance attributes includes a texture attribute in the form of image data.

[0069] This method includes:

[0070] Generate the first instance of the appearance model, which includes a first set of appearance attributes; and

[0071] Generate a second instance of the appearance model, which includes a second set of appearance attributes.

[0072] The appearance model includes a discrete texture table comprising multiple target textures, each represented by image data and associated with a different set of target coordinates indicating a specific combination of illumination and viewing directions.

[0073] The first and second instances of the appearance model are used to visualize the at least one virtual object.

[0074] While the simplified embodiment of the appearance model only describes the correlation between color and illumination and viewing conditions without considering spatial variations in appearance across the surface of an object, in this context, the appearance model is preferably a model that includes texture in addition to the angular correlation of reflection and / or transmission. Specifically, the appearance model can be a spatially varying two-way reflectance distribution function (“SVBRDF”) model, a two-way texture function (“BTF”) model, a two-way surface scattering distribution function (“BSSRDF” model”), a specialized model for automotive paint, etc. Many such models are known in the art.

[0075] Advantageously, the appearance model describes texture based on a discrete texture table. This texture table includes multiple textures, each represented by image data and associated with a different set of coordinates that indicates a specific combination of illumination and viewing directions.

[0076] Appearance models often require far more appearance attributes than the limited set of measured or predetermined appearance attributes actually available for the target material, candidate material, reference material, or test material, respectively. Furthermore, appearance models may require appearance attributes of different types than those available. In particular, the set of appearance attributes required for an appearance model can have a larger cardinality (i.e., contain a larger number of appearance attributes) than a limited set of available appearance attributes. In other words, the set of available appearance attributes may be sparse compared to the denser set of appearance attributes required for the appearance model. Mathematical operations, possibly involving transformations, fitting operations, and / or interpolation and extrapolation, may be necessary for instances generating appearance models from a limited set of available appearance attributes. For example, a limited set of available appearance attributes may contain color attributes (e.g., in the form of tricolor or spectral data) only for a limited number of illumination and observation direction pairs. This limited set of available appearance attributes may also include texture attributes (e.g., in the form of image data) only for a pair or a limited number of illumination and observation direction pairs. In contrast, appearance models can describe reflection and / or transmission as functions of angle and position in different forms. In particular, appearance models may require spectral and / or texture data for a much larger number of illumination and observation direction pairs than are available, and the required illumination and observation directions may differ from those available. Below, examples of how to generate instances of appearance models with a dense set of appearance attributes from a sparse set of available appearance attributes will be described in more detail.

[0077] Generating the first and / or second instance of the appearance model may include at least one of the following operations:

[0078] Interpolation between available appearance attributes under different combinations of illumination and observation directions; and

[0079] Extrapolate the available appearance attributes from the selected combination of illumination and observation directions.

[0080] Available appearance attributes may include multiple source textures, each associated with a different set of source coordinates, each set indicating a combination of illumination and viewing directions, for which the corresponding source texture indicates spatial variations in appearance. Source textures may be associated with source coordinate sets different from the target coordinates of a discrete texture table. Generating a first and / or second instance of the appearance model may then include determining at least one of the target textures by performing a statistical texture synthesis operation based on:

[0081] (i) The target coordinate set associated with the target texture;

[0082] (ii) Source material, and

[0083] (iii) Source coordinate group.

[0084] Each source and target coordinate set indicates a specific combination of illumination and observation directions. Each source and target coordinate set may, in particular, be a set of two or more angular coordinates. The source and target coordinates do not need to directly indicate the illumination and observation directions in a reference frame defined by the surface of the material. Instead, they can be derived from these directions through appropriate transformations. In particular, the source and / or target coordinates can be expressed using Rusinkiewicz parameterization [Rus98]. Specifically, in Rusinkiewicz parameterization, each source or target coordinate set may include polar angles or consist of polar angles.

[0085] Each source texture and each target texture can be represented by two-dimensional image data, typically in the form of an array of pixels, each pixel having a pixel value representing the reflection at that pixel's location. The source and target textures do not need to be the same size. Each source texture can represent a spatial variation of the material's reflective properties for a specific combination of illumination and viewing directions. Similarly, a target texture can represent a spatial variation of the material's reflective properties for another combination of illumination and viewing directions. This spatial variation can represent, for example, the effect of an effect pigment in the material, which produces a shimmering effect.

[0086] By performing a statistical texture compositing operation, a target texture is created that has similar statistical properties to the source texture, but is dissimilar to any source texture and the target texture at different target coordinates on a pixel-by-pixel basis. This mimics the properties of metallic paint.

[0087] Statistical texture synthesis operations may include:

[0088] (i) Assign interpolation weights to each element in the source texture based on the target and source coordinates; and

[0089] (ii) Use the source texture and the assigned interpolation weights to synthesize the target texture.

[0090] In some embodiments, assigning interpolation weights to each of the source textures includes the following process:

[0091] Create the Delaunay triangulation of the source coordinate group;

[0092] Find a simplex with multiple angles within the Delaunay triangulation containing the target coordinates; and

[0093] The target coordinates are used as the centroid coordinates of the found simplex as the interpolation weights of the source texture at the corner of the found simplex.

[0094] The interpolation weight of the source textures found outside the simplex is preferably set to zero, that is, the source textures found outside the simplex are not used to synthesize the target texture.

[0095] Synthesizing the target texture may include the following steps:

[0096] (a) Randomly select one of the source textures with a probability proportional to the interpolation weight of the source texture;

[0097] (b) Randomly extract texture patches from the selected source texture, where the texture patches are a part of the source texture;

[0098] (c) Modify the extracted texture by modifying the pixel values ​​in the extracted texture to obtain a modified texture, wherein the pixel value modification is performed in such a way that at least one statistical property of the modified texture approximates the corresponding average statistical property, and the average statistical property is determined by performing a weighted average on the source texture, which is weighted by interpolation weights.

[0099] (d) Insert the modified texture patch into the target texture so that the modified texture patch seamlessly fits the texture content already existing in the target texture; and

[0100] (e) Repeat steps (a)-(d) until the target texture is completely filled.

[0101] Each texture patch should be understood as a continuous portion of the source texture from which it has been extracted, i.e., a continuous image patch that has been cut out from the source texture. Each texture patch can have any shape. Rectangular shapes are preferred, and in particular square shapes. Each texture patch preferably has a side length that corresponds to at least twice the size of the pigment with the greatest effect in the paint being modeled (e.g., the largest reflective flake). On the other hand, preferably, each texture patch has an area not greater than 10% of the area of ​​the corresponding source texture.

[0102] Texture patches are extracted randomly, meaning the position of each extracted texture patch within the selected source texture is randomly determined, preferably with a constant weight over the area of ​​the source texture. By randomly extracting texture patches, it is ensured that the target textures obtained at different target coordinates are dissimilar.

[0103] In a preferred embodiment, the statistical property for the expected approximate match is a pixel value histogram. A pixel value histogram is a dataset that includes an indicator of the relative frequency of each of a range of brightness values ​​that may exist in an image. Pixel value histograms are widely used in digital image processing to visualize properties such as contrast and brightness of digital images. The generation of them from image data and their interpretation are well known in the art.

[0104] Modifying pixel values ​​preferably involves applying a pointwise transformation to the pixel value, i.e., a transformation applied individually to each pixel value, where the pointwise transformation is monotonically non-decreasing. In this way, it is ensured that pixel values ​​identical before modification remain identical after modification, and that the relative brightness order of pixels does not change; that is, a pixel value brighter than another pixel value before modification will not be darker than another pixel value after modification. Suitable transformation functions are well-known in the field of digital image processing. In particular, histogram matching algorithms for determining transformation functions that will match the histogram of one image with the histogram of another image are well-known in the art.

[0105] The modified texture piece is inserted into the target texture so that it seamlessly fits the texture content already existing in the target texture. This can be achieved using techniques well-known in the art, such as Graphcut / MinCut [Kw03]. In these techniques, a seam is calculated that enhances the visual smoothness between the existing texture content and the newly placed piece. The texture piece and the existing texture content are stitched together along this seam.

[0106] In addition to discrete texture surfaces, appearance models can include monochromatic luminance BRDF models to describe the angular correlation of overall reflectance properties averaged at surface location and wavelength. Instances of generating appearance models of materials can then include determining the parameters of the monochromatic luminance BRDF model.

[0107] The appearance model may also include a discrete color table with multiple entries in the form of color values, each entry associated with a specific set of coordinates, each set indicating the illumination and observation direction of the observed material. The color values ​​can be expressed in any color space, such as the RGB or CIEXYZ tri-color space, or any other color space such as CIELAB (L*a*b*), or in any format as spectral data representing the material's spectral response to incident light. Generating an instance of the appearance model can then include at least one of the following operations:

[0108] (i) Determining entries for the discrete color table by interpolation between available color attributes under different combinations of illumination and viewing directions; and / or

[0109] (ii) The entries for the discrete color table are determined by extrapolating the available color attributes from different combinations of illumination and observation directions.

[0110] Instead of a monochrome BRDF model associated with a color table, in alternative embodiments, the appearance model may include any other kind of model used to describe the angular correlation between brightness and color. For example, the appearance model may include a tri-color BRDF model that models the angular correlation of reflectance properties for each of the three distinct color channels in a tri-color color space (such as CIEXYZ or RGB), or that models the angular correlation of L*a*b* values ​​in the CIELAB color space.

[0111] The appearance model can also include a model with the effect of a transparent coating on top of an opaque or semi-transparent paint layer.

[0112] In some embodiments, the scene is visualized using defined illumination conditions, which, as is well known in the art, can be described, for example, by an environment map. The method may include changing the illumination conditions, particularly one or more illumination directions and / or the type of light source, i.e., the spectral properties of the light used for illumination, and visualizing the virtual object with the changed illumination conditions to facilitate comparison of the appearance of first and second materials under different illumination conditions. For example, in this way, metamerism can be detected before the production of test objects.

[0113] In some embodiments, the at least one virtual object is visualized in a defined direction, and the method includes changing the orientation of the virtual object, i.e., rotating the virtual object, and visualizing the virtual object in the orientation presented after the change in orientation. This allows for better discernment of subtle differences in appearance that depend on the direction of illumination and observation. The visualization can be performed in the form of video, which visualizes the continuously rotating virtual object. The method may also include changing the size and / or shape of the virtual object.

[0114] In some embodiments, the method includes:

[0115] Visualize the at least one virtual object together with virtual separator elements.

[0116] The at least one virtual object has first and second parts that are adjacent to each other, and the first and second parts are visually separated by the virtual separating element in such a way that the first and second parts appear to meet at the location of the virtual separating element.

[0117] The first instance of the appearance model is used to visualize the first part of the at least one virtual object, and

[0118] The second instance of the appearance model is used to visualize the second part of the at least one virtual object.

[0119] This method may include moving virtual divider elements relative to a virtual object. This can be done interactively, i.e., in response to user input, such as moving a pointing device like a mouse or trackpad, pressing a key on a keyboard, or waving a finger or pointing device like a digital pencil across a touchscreen. When the virtual divider element is moved, the first and second parts are permanently redefined in such a way that, when the virtual divider element is moved, the first and second parts still appear to meet at the location where the virtual divider element was. In this way, when the material of the virtual object changes, the user can visually assess how the appearance of the virtual object changes in the selected area of ​​the virtual object.

[0120] In some embodiments, the first and second materials may be substantially opaque materials, such as those used in the automotive industry for coating vehicle parts, or opaque plastics. In other embodiments, the first and second materials may be translucent materials, such as translucent plastics. The appearance model may be particularly suited to a specific class of materials. For example, for vehicle coatings, the appearance model may include a model of the effect of a transparent coating on top of an opaque or translucent paint layer and / or a model of the effect of reflective flakes in the paint layer. As another example, for translucent materials, the appearance model may use volumetric absorption and scattering coefficients, as well as phase function parameters, to model the subsurface light transport of the material in order to solve the radiative transfer equation.

[0121] The display device can be a screen, such as an LCD screen, a projector, or any other type of display device as is well known in the art. If the display device includes a screen, the screen can be touch-sensitive, as is well known in the art. The display device can be capable of creating 3D impressions, as is well known in the art. For example, the display device can be a VR headset or a 3D display. In such a case, the virtual dividing element can be a virtual plane. However, in a preferred embodiment, the display device visualizes the virtual object as a two-dimensional projection onto a two-dimensional viewing plane. In this case, the virtual dividing element can be a simple line, particularly a straight line, on the viewing plane.

[0122] This disclosure also provides a system for visualizing the appearance of at least two materials. The system includes a display device, at least one processor, and at least one memory, the memory including program instructions configured to use the display device to cause the processor to perform the methods described above. The system may also include the first and / or second appearance capturing devices described above, and may be accordingly configured to receive appearance attributes from these appearance capturing devices.

[0123] This disclosure also provides a computer program product including program instructions that, when executed by a processor, cause the processor to perform the methods described above. The computer program product may include a non-volatile computer-readable medium on which the program instructions are stored. The non-volatile medium may include a hard disk, a solid-state drive, a memory card, or any other type of computer-readable medium as is well known in the art.

[0124] In particular, program instructions may include rendering software configured to perform the following steps:

[0125] Receive the first and second sets of appearance attributes;

[0126] Receive three-dimensional geometric data representing the continuous three-dimensional surface geometry of the at least one virtual object; and

[0127] The scene, including the at least one virtual object, is rendered based on the first and second sets of appearance attributes and the three-dimensional geometric data.

[0128] The rendering software can be configured to receive first and second sets of appearance attributes in the form of first and second instances of the appearance model.

[0129] The program instructions may also include model generation software configured to perform the following steps:

[0130] Receive available appearance attributes; and

[0131] Based on the available appearance attributes, at least one of the following operations is used to determine the instance of the appearance model:

[0132] Interpolation between available appearance attributes under different combinations of illumination and observation directions; and

[0133] Extrapolate the available appearance attributes from the selected combination of illumination and observation directions.

[0134] The program instructions may also include formulation software configured to perform the following steps: using predetermined appearance properties associated with a plurality of reference materials and using a formulation to calculate appearance properties associated with a candidate material. The formulation software may also be configured to determine a formulation in such a way that the candidate material has expected appearance properties that at least approximately match the appearance properties of the target material. Attached Figure Description

[0135] Preferred embodiments of the invention are described below with reference to the accompanying drawings, which are for the purpose of illustrating the presently preferred embodiments of the invention and not for the purpose of limiting the presently preferred embodiments of the invention. In the drawings,

[0136] Figure 1 A schematic diagram illustrating a method for visualizing the appearance of two materials is shown;

[0137] Figure 2 The illustration is shown. Figure 1 Flowchart of the method;

[0138] Figure 3 A schematic diagram of an exemplary color mixing system is shown, representing a hardware-oriented illustration.

[0139] Figure 4 shows a perspective view of an exemplary appearance capture device according to the prior art;

[0140] Figure 5 shows a perspective view of the measurement array of the appearance capture device in Figure 4;

[0141] Figure 6 A diagram illustrating an exemplary discrete color table is shown;

[0142] Figure 7 An example texture of a discrete texture table is shown;

[0143] Figure 8 A diagram illustrating an exemplary discrete texture table is shown;

[0144] Figure 9 A schematic diagram of a method for generating a target texture based on multiple source textures is shown.

[0145] Figure 10 A schematic histogram of pixel values ​​is shown;

[0146] Figure 11 This diagram illustrates the insertion of a texture sheet into a target texture.

[0147] Figure 12 A flowchart illustrating a method for generating instances of appearance models is shown;

[0148] Figure 13 A flowchart is shown for a method for generating a target texture based on multiple source textures associated with different source coordinates;

[0149] Figure 14 A flowchart is shown for a method to generate a target texture based on multiple source textures associated with different components; and

[0150] Figure 15 This diagram illustrates how the information content varies across three dimensions. Detailed Implementation

[0151] definition

[0152] In this disclosure, references in the singular form may also include the plural. Specifically, unless the context otherwise indicates, the words “a” or “one” may refer to one or more.

[0153] The term "colorant" should be understood as a component of a material that provides the appearance of color when light is reflected from or transmitted through the material. Colorants include pigments and dyes. A "pigment" is a colorant that is generally insoluble in the base material. Pigments can be from natural or synthetic sources. Pigments can contain both organic and inorganic components. The term "pigment" also includes so-called "effect pigments" that produce special effects in materials. Examples include interference pigments and reflective particles or flakes. A "dye" is a colorant that is generally soluble in the base material.

[0154] The term "formulation" should be understood as a collection of information relating to determining how a material should be prepared. Materials can include coating materials (such as automotive paint), solid materials (such as plastic materials), semi-solid materials (such as gels), and combinations thereof. A formulation specifically includes the concentration of ingredients from which the material is composed, such as binders and colorants. Materials already prepared according to a formulation may also be referred to as a "formulation".

[0155] The term "visual appearance," or simply "appearance," should be broadly understood as the way an object reflects and transmits light, including but not limited to how individuals observing the object perceive its color and surface texture under various viewing conditions. Appearance also includes instrumental measurements of how an object reflects and transmits light.

[0156] One aspect of visual appearance is color. The “color” of an object is determined by the portion of the spectrum of incident white light that is reflected or transmitted but not absorbed. The color of an object can be described by “color attributes.” Generally, color attributes indicate the spectral response of an object when it is illuminated by incident light. In the context of this disclosure, the term “color attribute” is to be understood broadly as including any form of data indicating the spectral response of an object when it is illuminated by incident light. Color attributes can take the form of color values ​​in any color space, such as in a three-color space like RGB or CIEXYZ, or in any other color space like CIELAB (L*a*b*), or in the form of spectral data representing the spectral response of a material to incident light in any format. In the context of this disclosure, color attributes may in particular include reflectance values ​​and / or absorption and scattering coefficients of a material at multiple wavelengths. A “discrete color table” is to be understood as involving a set of multiple sets of color attributes, each set of color attributes being associated with different combinations of illumination and viewing directions.

[0157] Another aspect of visual appearance is texture. The term "texture" is broadly understood to refer to spatial variations in the appearance of a material's surface, both at microscopic or mesoscopic scales (i.e., scales at which individual structural elements are typically not perceptible to the naked eye) and at macroscopic scales (i.e., scales at which individual structural elements are perceptible to the naked eye). Texture, as understood in this disclosure, includes phenomena such as roughness, luster, and surface morphology. Texture can be described by "texture properties." In the context of this disclosure, the term "texture property" is broadly understood to include any form of data capable of quantifying at least one aspect of texture. Examples of texture properties include global texture properties, such as global roughness parameters or global luster parameters. In some embodiments, texture properties may include normal maps or height maps. In some embodiments, texture properties may include image data. In some embodiments, image data may be associated with a specific combination of illumination and viewing directions. In such embodiments, texture properties may include multiple image datasets, each associated with a different combination of illumination and viewing directions. A “discrete texture table” should be understood as a collection of multiple sets of texture attributes, preferably in the form of image data, each set of texture attributes being associated with different combinations of illumination and viewing directions. In some embodiments of this disclosure, the images in the discrete texture table are generated from a set of source textures, and these images are therefore referred to as “target textures”.

[0158] The term "appearance model" should be understood as a formal construction involving the mathematical description of appearance using multiple material-related parameters called "appearance attributes." Appearance attributes can include color attributes and texture attributes. The appearance model is preferably independent of the apparatus and platform; that is, it is independent of the specific measuring device that may have been used to determine the appearance attributes, and it is independent of the specific rendering platform used for visualization. The appearance model provides a mathematical description of the appearance in such a form and with such a level of completeness that it is possible to generate visualizations (i.e., rendering and display) of virtual objects under arbitrary lighting and viewing conditions using the appearance model in conjunction with the geometric model of the virtual object. For any part of the surface of the virtual object whose geometry is defined by the geometric model, and for any given lighting and viewing angle, the appearance model provides the necessary information to calculate the appropriate appearance attributes.

[0159] A "virtual object" is an object that exists only virtually in a computer. A virtual object may or may not correspond to a real, tangible object. Virtual objects can be mathematically defined through geometric models and associated appearance information.

[0160] A "geometric model" of a real or virtual object is to be understood as at least an approximate representation of the geometry of any surface of that object in three dimensions. For example, in some embodiments, the geometric model defines curves along at least two mutually orthogonal directions. In other embodiments, the geometric model defines multiple polygons or faces. The geometric model may be represented, for example, by a CAD file.

[0161] The term "instance of appearance model" should be understood as a set of values ​​for all appearance attributes relating to a particular appearance model, supplementing information that allows identification of the underlying appearance model. In other words, while the appearance model itself defines the formal construction of how an appearance is described based on a set of appearance attributes, instances of the appearance model include, for example, actual values ​​of these appearance attributes specific to a given material, such as those determined by measurement and / or derived from appearance attributes of individual components of the material. In particular, instances of the appearance model can be provided in the form of data files. Preferably, the data files are in a device and platform-independent format, such as AxF proposed by X-Rite. TM Format. Therefore, an AxF file can be considered a representation of an "instance" of the appearance model. However, this disclosure is not limited to a specific format for instances of the appearance model, and instances of the appearance model may be provided in another file format (e.g., MDL format), or it may even be provided in a form different from a file, such as as a data stream.

[0162] Luminance properties can be represented in an appearance model with a bidirectional reflectance distribution function (BRDF). A BRDF is generally understood as a function defining how light is reflected at an opaque surface depending on the direction of illumination and observation, thus providing the ratio of reflected radiation emitted along the observation direction to the irradiance incident on the surface from the direction of illumination. If the surface exhibits spatial variation in this ratio, the BRDF is understood to provide an average of this ratio over a region of the surface. A monochromatic luminance BRDF is a BRDF that provides a (possibly weighted) average of the ratio across all visible wavelengths, thus simulating the overall luminance variation of the surface.

[0163] "Image data" is data representing an image. An image includes images of an actual target surface or object, as well as composite images derived from one or more geometric models combined with one or more sets of appearance attributes. An image may take the form of a two-dimensional array of image elements ("pixels"), each pixel having a specific pixel value. Pixel values ​​may represent the reflection at the pixel's location at a specific wavelength, the average reflection within a specific wavelength range, or the average reflection across all visible wavelengths. Therefore, in some embodiments, image data may be provided as an array of pixel values. In other embodiments, image data may be provided in a compressed or transformed form. An "image dataset" is image data comprising at least one image or a dataset consisting of image data of at least one image, i.e., a dataset representing one or more images.

[0164] If the pixel values ​​of two images X and Y of the same size are uncorrelated, they are considered "dissimilar" on a pixel-by-pixel basis. Dissimilarity can be quantified by a correlation measure of pixel values ​​that is appropriately defined. For the purposes of this disclosure, if the Pearson product-moment correlation coefficient r of pixel values ​​in an image... XY If the absolute value of the correlation coefficient is no greater than 0.1, preferably no greater than 0.02, then the images can be considered "dissimilar". The Pearson product-moment correlation coefficient r... XY The quotient is defined as the product of the covariance of pixel values ​​and the standard deviation (i.e., the square root of the variance) of the pixel values ​​of the two images:

[0165]

[0166] Here, x i Represents the pixel values ​​in image X. y represents the arithmetic mean of the pixel values ​​in image X. i Represents the pixel value in the Y region of the image. This represents the arithmetic mean of the pixel values ​​in image Y, where N indicates the number of pixels in each image (which is the same for both images since they are the same size), and the summation is performed over all pixels. The arithmetic mean of the pixel values ​​in image X. Defined in the usual way

[0167]

[0168] The "Rusinkiewicz parameterization" describes the illumination and observation directions based on the "halfway vector" and the "difference vector." The "halfway vector" is defined as the vector midway between the incident and reflected rays, and the "difference vector" is the illumination direction in a reference frame where the halfway vector is located at the north pole. The spherical coordinates of the halfway vector in the sample-fixed reference frame where the surface normal is located at the north pole can be expressed as (θ... h φ h ), where θ h This is called the "midpoint" in polar coordinates. The difference vector, in which the midpoint vector lies, can be represented in spherical coordinates in a reference frame where the midpoint vector is located at the North Pole as (θ). I φ I ), where θ I This is called the polar coordinate "difference angle". The remaining angles defining the directions of illumination and observation are the azimuth angles φ. h and φ I For details, please refer to [Rus98], the contents of which are incorporated herein by reference in their entirety.

[0169] In some embodiments of this disclosure, barycentric coordinates are used to determine texture properties associated with an appearance model. "Barycentric coordinates" are coordinates in a coordinate system where the position of a point on a simplex (i.e., a triangle in the case of two dimensions) is specified by weights assigned to its vertices. In two dimensions, a simplex is a triangle. The two-dimensional barycentric coordinates of point r relative to vertices r1, r2, and r3 are given by three numbers λ1, λ2, and λ3, such that...

[0170] (λ1+λ2+λ3)r=λ1r1+λ2r2+λ3r3,

[0171] In this context, at least one of the numbers λ1, λ2, and λ3 must be non-zero. It may be necessary that the barycentric coordinates are non-negative (i.e., that the point may be located within the convex hull of a vertex). It may also be necessary that the barycentric coordinates satisfy the condition λ1 + λ2 + λ3 = 1. These are then referred to as "absolute barycentric coordinates".

[0172] An "appearance capture device" is a device capable of determining one or more appearance attributes of an object. Depending on the appearance attribute to be determined, the appearance capture device may take the form of, for example, a camera, a colorimeter, a spectrophotometer, or an imaging spectrophotometer.

[0173] A spectrophotometer is a device used to determine the reflectance and / or transmission properties (as a function of wavelength, i.e., the spectral response of the object) of a surface or material when illuminated with visible light. Different types of spectrophotometers are known, each with different geometries and optimized for different purposes. One important type is the integrating sphere spectrophotometer. An integrating sphere spectrophotometer comprises an integrating sphere, a hollow spherical cavity defined by a diffuse white inner surface, having at least one inlet port for illumination and at least one outlet port for observation. The integrating sphere produces a uniform scattering or diffusion effect. Through multiple scattering and reflection, light incident on any point on the inner surface is uniformly distributed to all other points. The influence of the original direction of light is minimized. Examples of integrating sphere spectrophotometers are X-Rite models Ci7860 and Ci7500. Other types of spectrophotometers determine only spectral information for a single narrow range of illumination direction (e.g., 45° to the surface normal) and a single narrow range of observation direction (e.g., 0° to the surface normal). Examples include models 962 and 964, available from X-Rite. Several other spectrophotometers, referred to as "gonometric spectrophotometers" or "multi-angle spectrophotometers," are capable of determining spectral information for multiple combinations of different illumination and observation directions. "Imaging spectrophotometers" additionally possess imaging capabilities, meaning they can include one or more cameras to capture one or more digital images of an object. Examples of multi-angle spectrophotometers with imaging capabilities include the benchtop model TAC7 or the handheld models MA-T6 or MA-T12, available from X-Rite.

[0174] Materials can be transparent, translucent, or opaque. A material is "transparent" if it allows light to pass through without significant absorption and scattering. A material is "translucent" if it allows light to pass through, but the light may be scattered at either of the two interfaces or internally. A material is "opaque" if it does not transmit light. A material may be opaque only in some spectral regions, while being translucent or transparent in others, and vice versa. For example, a material may strongly absorb red light (i.e., it is essentially opaque to red light) and only weakly absorb blue light (i.e., it is transparent to blue light). Some more complex materials, especially those with variable angular appearance, may include combinations of transparent, translucent, and opaque materials. For example, a paint coating may include an opaque base layer and a transparent topcoat. Opaque (reflective or interfering flakes) or translucent pigments may be included in opaque, transparent, or translucent layers of a paint coating.

[0175] For the purposes of this disclosure, a material will be broadly considered "translucent" if a relatively thin slice transmits a considerable portion of the incident radiant flux in at least a portion of the visible spectrum, for example, if a slice with a thickness of 0.1 mm transmits at least 1% of the incident radiant flux in at least a portion of the visible spectrum. In this sense, the term "translucent" includes the term "transparent," that is, for the purposes of this disclosure, transparent materials are also considered translucent. Examples of translucent materials in this sense include many common polymer-based plastic materials, including but not limited to organic polymers (such as PET, PP, PE, PMMA, PS, PC, PVC, PTFE, nylon), organic copolymers (such as styrene-butadiene copolymers), inorganic polymers (such as polysiloxanes), and many natural polymers. Translucent plastic materials may contain pigments and additives. However, for the purposes of this disclosure, other classes of materials may also be translucent, including, for example, silicate glass or paper.

[0176] For the purposes of this disclosure, a material is to be understood as “homogeneous” if its subsurface light transport properties remain unchanged at macroscopic or mesoscopic scales, for example, at scales greater than 1 μm. In particular, homogeneous materials do not include objects with varying appearances at mesoscopic or macroscopic angles, such as thin sheets.

[0177] The term "macroscopic surface geometry" should be understood to refer to the overall geometry of a product, excluding micro or mesoscopic surface structures, i.e., variations in surface geometry at micro or mesoscopic scales below, for example, 1 mm. For example, a local variation of surface height with a local average of less than, for example, 1 mm can be considered a micro or mesoscopic surface structure, and therefore the macroscopic surface geometry can be equivalent to the surface geometry averaged over a length scale of at least 1 mm. In mathematical terms, if the surface geometry at least approximately and at least locally corresponds to a two-dimensional differentiable manifold in three-dimensional Euclidean space, then the surface geometry is "continuously curved."

[0178] The term "rendering" refers to the automated process of generating a photorealistic image of a scene using a computer program. In this disclosure, the scene includes at least one virtual object. Input information for the rendering operation includes a 3D geometric model of at least one virtual object, at least one set of appearance attributes associated with the virtual object, information about the position and orientation of at least one virtual object in the scene, lighting conditions (which may take the form of an environment map), and parameters characterizing the observer, such as viewpoint, focal length, field of view, depth of field, aspect ratio, and / or spectral sensitivity. The output of the rendering operation is an image of the scene, which includes at least a portion of the virtual object. Many different rendering algorithms are known at varying levels of complexity, and the software used for rendering can employ a variety of different techniques to obtain the final image. Tracking every single light particle in a scene is generally impractical because it requires excessive computational time. Therefore, simplified techniques for modeling light transport, such as ray tracing and path tracing, are typically used…

[0179] The term "visualization" encompasses rendering and displaying a scene that includes virtual objects. For this display, a display device is used. The term "display device," or simply "monitor," is to be understood as referring to a computer output device used to present information in a visual form. A display device can take the form of a computer monitor, TV screen, projector, VR headset, or the screen of a handheld device such as a smartphone or tablet. A display device can be a touchscreen. In some embodiments, the display device can be a "virtual lightbooth" as disclosed in EP 3 163 358 A1 to provide a particularly realistic impression of the rendered scene.

[0180] The term "database" refers to a collection of organized data that can be accessed electronically by a computer system. In a simple embodiment, a database can be a searchable electronic file in any format. Examples include Microsoft Excel. TM A spreadsheet or a searchable PDF document. In more complex embodiments, the database can be a relational database maintained by a relational database management system using a language such as SQL.

[0181] The term "computer" or "computing device" refers to any device that can be instructed by a program to automatically perform a sequence of arithmetic or logical operations. Without limitation, a computer can take the form of a desktop computer, laptop computer, tablet computer, smartphone, programmable digital signal processor, etc. A computer typically includes at least one processor and at least one memory device. A computer can be a subunit of another device (such as an appearance capture device). A computer can be configured to establish a wired or wireless connection to another computer, including a computer used for querying databases. A computer can be configured to be coupled to a data input device (such as a keyboard or computer mouse) or a data output device (such as a monitor or printer) via a wired or wireless connection.

[0182] The term "computer system" should be broadly understood to include one or more computers. If a computer system includes more than one computer, these computers do not necessarily need to be located in the same place. Computers within a computer system can communicate with each other via wired or wireless connections.

[0183] A processor is an electronic circuit that performs operations on external data sources, particularly memory devices.

[0184] A "memory device," or simply "memory," is a means for storing information for use by a processor. Memory devices can include volatile memory, such as random access memory (RAM), and non-volatile memory, such as read-only memory (ROM). In some embodiments, a memory device can include a non-volatile semiconductor memory device, such as (E)EPROM or flash memory, which may take the form of, for example, a memory card or a solid-state drive. In some embodiments, a memory device can include a high-capacity storage device with mechanical components, such as a hard disk. A memory device can store programs for execution by a processor. Non-volatile memory devices can also be referred to as non-volatile computer-readable media.

[0185] A program is a collection of instructions that can be executed by a processor to perform a specific task.

[0186] A "wired connection" is a connection via an electrical conductor. A wired connection may include one or more cables. A "wireless connection" is a connection involving the electromagnetic transmission of information between two or more points that are not connected by an electrical conductor. Wireless connections include those via WiFi. TM ,Bluetooth TM Connections to 3G / 4G / 5G mobile networks, optical communication, infrared, etc.

[0187] Exemplary embodiments of the method using color matching software

[0188] Figure 1An exemplary embodiment of a method for visualizing the appearance of two or more materials in a vehicle repair context is illustrated.

[0189] Suppose a damaged vehicle needs repair at a body shop. Technicians at the body shop use a handheld appearance capture device 52 to determine a set of appearance properties 54 of the paint coating on an intact vehicle part. The paint coating is an example of a target material, and the intact vehicle part is an example of a target object containing the target material. The appearance properties 54 of the paint coating include, on the one hand, color properties in the form of spectral data for multiple pairs of illumination and viewing directions, and on the other hand, texture properties in the form of image data for multiple pairs (potentially different pairs) of illumination and viewing directions.

[0190] The appearance capture device 52 transmits the measured appearance attributes 54 to a computer system. The computer system may include a local client computer located at the body repair shop. The computer system may further include one or more remote computers located at one or more locations different from the body repair shop. The local client computer may be, for example, a mobile electronic device, such as a laptop or tablet. The remote computer may act as a server for the local client computer at the body repair shop.

[0191] Several elements of the computer system execute software. In a first aspect, the computer system executes model generation software 102. The model generation software 102 generates a first instance 56 of a selected form of appearance model based on measured appearance attributes 54, the first instance 56 representing the appearance of the target material. The first instance of the appearance model is stored in a first AxF file in a form independent of the device and platform.

[0192] In the second aspect, the computer system executes color formulation software 104. Color formulation software 104 determines one or more candidate formulations 60 from a database of reference formulations. Each candidate formulation defines a candidate material (in this example, a candidate paint coating) whose appearance properties may match the measured appearance properties 54 of the target material. Candidate materials include one or more colorants in the base formulation. To determine candidate formulations, formulation software 104 retrieves predetermined appearance properties associated with different reference materials from database 106. Specifically, predetermined appearance properties may be associated with individual colorants and / or with reference formulations containing one or more colorants dispersed in the base formulation. Specifically, database 106 may include two sub-databases: a colorant database storing appearance properties associated with individual colorants and base materials, and a formulation database storing appearance properties associated with reference formulations. The appearance properties in database 106 may have been predetermined by performing measurements on reference objects, including reference materials made according to reference formulations or components comprising reference formulations.

[0193] For example, to determine appearance properties associated with individual colorants in a colorant database, a reduction in the coating of a formulation containing individual colorants at different concentrations can be prepared and measured; to determine appearance properties in a formulation database, a reduction in the coating of a formulation prepared according to a reference formulation can be measured. An appearance capturing device can be used to perform the measurement, which can be of the same type as appearance capturing device 52, or it can be of a different type, as will be discussed in more detail below.

[0194] The formulation software uses predetermined appearance attributes retrieved from database 106 to calculate candidate formulations whose associated appearance attributes are expected to "closely approximate" the measured appearance attributes of the target material. If the appearance difference between the candidate formulation's appearance attributes and the measured appearance attributes is small according to a predefined difference norm, then the candidate formulation's appearance attributes are "closely approximate" the measured appearance attributes of the target material. In other words, the formulation software performs a minimization algorithm to determine candidate formulations that are close to the minimum of the difference norm. The formulation software 104 outputs one or more of the candidate formulations 60 thus determined. A technician selects and optionally manually modifies one of the candidate formulations. The formulation software 104 provides a set of candidate appearance attributes 64 for the selected (and optionally modified) candidate formulation 60. In some use cases, the candidate formulation corresponds to a single reference material, i.e., the candidate appearance attributes include appearance attributes associated with a single reference material. In other use cases, the candidate appearance attributes are calculated based on measured appearance attributes associated with multiple reference materials. Reference materials may include individual components of the candidate formulation in a manner that allows determination of the appearance attributes associated with these individual components. This is particularly useful when modifying a candidate formulation by changing the proportions of individual components. The model generation software 102 receives the candidate appearance attributes 64 and generates a second instance 66 of the appearance model, representing the expected appearance of the paint coating prepared according to the candidate formulation 60. The second instance 66 of the appearance model is stored in a second AxF file.

[0195] In the third aspect, the computer system executes rendering software 108. The rendering software renders a virtual object 72, that is, it creates a photorealistic digital image of the virtual object 72 based on a geometric model of the surface geometry of the virtual object 72, at least one instance of the appearance model, and lighting and viewing conditions. The rendered virtual object 72 is displayed in a scene on a display 70. The geometric model defines a continuous three-dimensional macroscopic surface geometry having surface normals distributed over relatively large solid angles; that is, it includes curved or straight surface portions having directions perpendicular to the surface portions pointing in many different directions. In 3D computer graphics, polygon modeling is well known as a method for modeling objects by using polygon meshes to represent or approximate their surfaces. These are also considered continuously curved three-dimensional macroscopic surfaces if the polygon meshes substantially appear as continuously curved surfaces during rendering. The rendering software generates a two-dimensional image of the virtual object 72 in a specific orientation and under specific lighting conditions (assuming specific viewing conditions). The surface geometry, orientation, and illumination conditions are selected in such a way that the rendered image gives a good impression of the appearance of the virtual object with respect to a wide range of angles between the surface normal, the illumination direction, and the viewing direction, so as to allow the observer to evaluate the appearance of the rendered virtual object 72 with respect to a wide range of these directions.

[0196] Virtual object 72 has first and second parts 72a and 72b that are adjacent to each other. The first part 72a is rendered using a first instance 56 of the appearance model, while the second part 72b is rendered using a second instance 66 of the appearance model. A virtual dividing line 74 is visualized between the first and second parts 72a and 72b. The first and second parts 72a and 72b appear to meet at the virtual dividing line 74.

[0197] Display 70 can be located at the auto repair shop. For example, display 70 could be a touchscreen display of a local client computer at the auto repair shop. A technician at the auto repair shop can use their finger or a pointing device such as a digital pen, trackpad, or mouse to move a virtual line 74 across display 70 and observe how the appearance matches or differs between first and second instances of an appearance model rendered on a virtual object 72; that is, how the appearance matches or differs between the measured appearance of the actual paint coating on the vehicle and the expected appearance of a paint coating prepared according to a candidate formulation. Optionally, the technician can change the shape of the virtual object, rotate the virtual object in space, and simultaneously change the illumination and viewing angles. Optionally, the technician can change the illumination conditions, including the direction of illumination and the selection of the light source.

[0198] In some embodiments, the virtual object 72 may have the shape of an actual part of the damaged vehicle, such as the shape of the target object 50. For this purpose, the rendering software 108 can retrieve a three-dimensional geometric model corresponding to the geometry of the vehicle part from a suitable memory (e.g., from a database 110 storing geometric data of multiple vehicle parts). For this purpose, a technician can provide, for example, the manufacturer and model of the damaged vehicle or other vehicle information such as a unique vehicle identification number, and one or more components to be rendered as the virtual object 72.

[0199] In other embodiments, the rendered virtual object 72 is a three-dimensional object that differs from the actual car part to be repaired, but has a three-dimensional shape useful for examining the appearance properties of various materials. Also in such embodiments, the rendering software 108 can retrieve the three-dimensional geometric model of the virtual object from memory, for example, from database 110. Regardless of whether the virtual object 72 represents an actual car part, the virtual object 72 preferably allows for simultaneous examination of color and texture differences at a large number of angles between surface normals, illumination directions, and viewing directions.

[0200] When viewed on monitor 70, the appearance of the first part 72a of the rendered virtual object does not need to perfectly match the appearance of the actual target object 50. In particular, monitor 70 does not need to be calibrated. It is important that the appearances of the first and second parts 72a and 72b are directly comparable to each other in both color and texture. This is possible even if the colors on the monitor are not true colors. By using the same appearance model for both parts, the direct comparability of the appearances of the two parts 72a and 72b is ensured.

[0201] It should also be noted that the appearance model itself and the rendering software do not need to be perfect to ensure direct comparability. The appearance model and rendering software are sufficient to make a reasonably realistic judgment on the differences between two instances representing the actual coating material on the target object 50 and the coating material according to the candidate formulation 60.

[0202] If a visual comparison of the first and second parts 72a, 72b shows an unsatisfactory match, the technician can modify candidate recipe 60 in formulation software 104 by selecting a different candidate recipe or by modifying a previously selected candidate recipe. Formulation software 104 provides the appearance attributes of the modified candidate recipe to model generation software 102, which creates a second instance of the modified appearance model based on these attributes. Rendering software 108 can be instructed to render the second part of the virtual object using the modified second instance of the appearance model, thereby replacing the previous candidate recipe in the visualization with the modified recipe, or the rendering software can be instructed to divide the virtual object into three parts 72a, 72b, 72c to visualize the reference material along with the two candidate recipes, thereby adding another movable dividing line 76 (“recursive divider control”) to the visualization.

[0203] Once a satisfactory match has been achieved, an actual test object 80 can be produced using the selected candidate formulation 60, for example, a reduced version. The appearance attributes of the test object 80 can be determined using the appearance capture device 82. The appearance attributes of the test object 80 can be compared with those of the target object 50 to determine whether the match is objectively within tolerance. This can be done by evaluating an appropriate difference norm between the measured appearance attributes of the test object 80 and those of the target object 50. Additionally or alternatively, a third instance of the appearance model can be generated using the appearance attributes of the test object 80, and the target material and the test material can be visualized side-by-side on the display 70 using the associated instance of the appearance model. Of course, if both the physical test object 80 and the physical target object 50 are located in the same location, a direct comparison of the two objects is also possible. If the match is still not within tolerance, the formulation software 104 can further modify the candidate formulation 60 by considering the differences between the predicted appearance attributes of the candidate formulation 60 and those appearance attributes already actually determined for the test object 80.

[0204] Test subject 80 can be manufactured by technicians in a body repair shop. However, most often, test subject 80 will be manufactured by a paint supplier located far from the body repair shop. Therefore, the appearance capture device 82 used to determine the appearance attributes of test subject 80 may differ from the appearance capture device 52 used to determine the appearance attributes of target object 50.

[0205] In the above embodiment, it is assumed that a technician at the auto repair shop not only operates the appearance capture device 52 to determine a set of appearance attributes 54 of the target object 50, but also operates the formulation software 104 to define one or more candidate formulations and uses the display 70 to compare the first and second parts 72a, 72b of the virtual object 72. However, these tasks can also be distributed among different people working at different locations. For example, the technician at the auto repair shop can transmit the appearance attributes 54 to a remote paint supplier, and a paint expert at the paint supplier can use the formulation software 104 to define candidate formulations and compare different parts of the virtual object to determine whether the expected match is satisfactory. Therefore, the computer system can include at least one computer under the control of the paint supplier, such as a computer executing the formulation software 104, model generation software 102, and rendering software 108. Many other possibilities exist for the division of tasks among the different entities involved in the process. Therefore, it will be apparent to the technician that the computer system can include several computers that may be located in different locations and may be under the control of different entities, communicating via wireless or wired data communication connections.

[0206] Other possible ways to visualize objects using instances of appearance models

[0207] In alternative embodiments, instances of the appearance model can be used to visualize scenes other than those comprising only a single virtual object, which has two or more parts visualized using different instances of the appearance model. For example, a scene comprising two or more identical or different virtual objects, each of which is visualized using a different instance of the appearance model. Generally, a scene comprises one or more virtual objects, and different instances of the appearance model are used to visualize different parts of the scene. For example, two objects of the same shape (e.g., two vehicle parts of the same shape, such as rearview mirrors) can be visualized side-by-side using first and second instances of the appearance model combined with a common geometry model.

[0208] flow chart

[0209] Figure 2An exemplary flowchart illustrating a method for visualizing the appearance of two materials is shown. In step 501, a user (e.g., a technician at a car body repair shop) determines the appearance attributes 54 of a target object 50 by measuring the light reflection and / or transmission properties of the target object, and optionally texture properties, using a first appearance capture device 52. In step 502, the appearance capture device 52 transmits these measured appearance attributes to a component of a computer system, such as a handheld computer in the car body repair shop. In step 503, the computer system uses model generation software 102 to generate a first instance 56 of the appearance model from the measured appearance attributes. In step 504, the computer system reads the appearance attributes of one or more reference materials from a database 106. In step 505, the computer system uses formulation software 104 to determine one or more candidate formulations 60. The formulation software 104 may perform step 505 by simply selecting one or more reference materials having appearance attributes similar to the measured appearance attributes 54 of the target object 50, by modifying the formulation of the retrieved reference materials, or by generating a new candidate formulation 60 if no close match is found in the database 106. In step 506, the computer system uses formulation software 104 to determine the appearance attribute 64 associated with candidate formulation 60. In step 507, the computer system uses model generation software 102 to generate a second instance 66 of the appearance model from the appearance attribute 64 associated with candidate formulation 60. In step 508, the computer system uses rendering software 108 to visualize virtual object 72 based on the two instances 56, 66 of the appearance model, the geometric model of virtual object 72, and lighting and viewing conditions. In step 509, the user compares the rendered virtual object portions 72a, 72b to obtain an acceptable appearance match. In step 510, if the visualized virtual object portions 72a, 72b do not provide an acceptable visual match, the user can modify the formulation or select a different candidate formulation 60. Once an acceptable rendering match is obtained, in step 511, the user prepares test object 80 using the candidate formulation. In step 512, the same or different users use a second appearance capture device 82 to determine the appearance attributes of test object 80. In step 513, the appearance attributes of the test subject 80 are transmitted to the computer system, and the computer system uses formulation software 104 to determine the modified formula, and refine the match if necessary. Steps 506 to 513 can then be repeated for the modified formula.

[0210] Computer System: Exemplary Hardware

[0211] Figure 3 The diagram shows that it can be used Figure 1 and 2An exemplary hardware-oriented block diagram of the computer system used in the method illustrated herein. In this example, the computer system includes two main components: a local client computer (e.g., a laptop or tablet computer) 300, which may be located at a vehicle repair shop, and a remote server computer 360, which may be located at a paint supplier's premises.

[0212] As is well known in the art, various components of the local client computer 300 communicate with each other via one or more buses 301. The local client computer 300 includes one or more processors 310. As is well known in the art, processors 310 may include, for example, a single-core or multi-core CPU and a GPU. The local client computer 300 also includes one or more non-volatile memory devices 320, such as flash memory devices and / or hard disk drives. The non-volatile memory 320 stores, in particular, the operating system 321 of the local client computer 300 and several applications, including model generation software 102 and rendering software 108. The non-volatile memory 320 also stores user data. The local client computer 300 also includes random access memory (RAM) 330, and input / output (I / O) interfaces 340 and communication interfaces 350. Attached to the communication interface are a display device 70 and a pointing device 90. The communication interface 350 may include, for example, an Ethernet interface, a WiFi interface, or Bluetooth. TM One or more of the following: interfaces, etc. A communication interface can be used to communicate with a remote 360 ​​server.

[0213] The remote server computer 360 can be configured similarly to the client computer 300. It stores configuration software 104 for execution. The client computer 360 may include or be connected to the database 106. Communication between the server computer 360 and the client computer 300 can be conducted via wired or wireless networks, such as via LAN or WAN, and particularly via the Internet.

[0214] The client computer also communicates with the first and / or second appearance capture devices 52, 82 via communication interface 350.

[0215] In other embodiments, some functions of the client computer 300 are instead transferred to the server computer 360. Specifically, the server computer (rather than the client computer) can execute the model generation software 102 and / or rendering software 108. This can be useful if the client computer is a "thin" client with limited computing power.

[0216] In some other embodiments, the computer system consists of only a single computer that executes all of the components described above in the software components.

[0217] Exemplary appearance capture device

[0218] In the example above, appearance capture devices 52 and 82 are preferably multi-angle spectrophotometers with imaging capabilities. Such devices are known in themselves. For example, the MA-TX series available from X-Rite can be used.

[0219] Figures 4 and 5 illustrate exemplary handheld appearance capture devices that can be used in conjunction with the present invention. The appearance capture devices of Figures 4 and 5 are described in more detail in document US20140152990A1, the contents of which are incorporated herein by reference in their entirety for the purpose of teaching handheld appearance capture devices.

[0220] An appearance capturing device is configured to capture the visual impression of a measured object. Hereinafter, the appearance capturing device is also referred to as a “measuring device” or simply “device.” Hereinafter, the term “measuring array” is understood to refer to the sum of the components of a handheld measuring device used to illuminate measuring points on the surface of the measured object, capture the light reflected from those measuring points, and convert it into a corresponding electrical signal. The term “device normal” is understood to refer to an imaginary straight line that is fixed relative to the device and extends substantially through the center point of the measuring opening of the measuring device, and is perpendicular to the surface of the measured object when the measuring device is positioned on the planar surface of the measured object. The plane of the measuring opening is generally parallel to the surface of the measured object, such that the device normal is also perpendicular to the measuring opening. The term “vertical” is understood to refer to the direction of the device normal. Therefore, a vertical section is understood to refer to a planar section in a plane containing the device normal or parallel to the device normal. In the following description of the measuring device, the direction and / or angle are relative to the device normal, which is spatially fixed relative to the measuring device.

[0221] The handheld measuring device shown in Figure 4 is indicated as a whole by reference numeral HMD. It includes a housing H housing the measuring array, as shown in Figure 4, and an electronic control array (not shown) for controlling the measuring array. Two gripping parts 1 and 2 are implemented laterally on the housing H. A wrist strap 3 is arranged on the upper side of the housing H. A display array 4 is provided on the front side of the housing H. An operating member (not shown) is arranged on the upper side of the housing H.

[0222] The lower side of the housing H includes a housing base 5 reinforced by a substrate 7, which is equipped with a measurement opening 6. The housing base 5 includes a hole (not indicated by a reference mark) in the area of ​​the measurement opening 6, allowing light to exit the interior of the housing through the hole and the measurement opening 6, and conversely, allowing light from the outside to enter the interior of the housing through the measurement opening 6 and the hole. Three support members 7a, 7b, and 7c are arranged around the measurement opening 6 on the substrate 7 and help to ensure proper positioning of the measuring device even on curved measuring surfaces, such that the device normal is completely or at least largely coincident with the normal of the measuring surface at the center point of the measurement point.

[0223] The device normal is indicated by reference mark DN in Figure 4. It is perpendicular to the substrate 7 and extends through the center point of the measuring opening 6.

[0224] The arrangement of the measurement array is illustrated in Figure 5. It includes an arcuate body 10, which is fixedly held within a housing H, and the optical and / or optoelectronic components of the measurement array are arranged within the arcuate body 10. In the exemplary embodiment shown, these components include seven illumination devices 21, 22, 23, 24, 25, 26, and 27, and three pick-up devices 31, 32, and 33. Furthermore, an illumination device 28 for diffuse illumination is provided adjacent to the measurement opening 6.

[0225] Seven irradiation devices 21 to 27 irradiate measurement points on the surface of the object being measured along different irradiation directions relative to the device normal DN. For example, the optical axes of the irradiation devices 21 to 27 can be oriented at angles of -60°, -45°, -30°, -20°, 0°, +30°, and +65° relative to the device normal. All seven irradiation devices 21 to 27 are arranged such that their optical axes lie in a common plane containing the device normal DN, which is referred to below as the system plane SP.

[0226] Two of the three pickup devices 31 to 33 are implemented as spectral measurement channels; the third pickup device is implemented as a spatially resolved color measurement channel. The pickup devices receive measurement light reflected from the region of the irradiated measurement point of the object being measured at observation angles of +15° and +45° in the system plane SP. The two pickup devices 31 and 32 forming the spectral measurement channel include two spectrometers 31a and 32a, to which the measurement light is fed via lenses and optical fibers 31c and 32c. The pickup device 33 forming the spatially resolved measurement channel includes a color-enabled (RGB) camera 33a to which the measurement light is applied via a beam splitter and a lens (not shown). The beam splitter is located in the beam path of pickup device 32 and laterally guides a portion of the measurement light from the arcuate body 10 onto camera 33a at an observation angle of +15°. Therefore, pickup devices 32 and 33 share the measurement light and receive it at exactly the same observation angle.

[0227] The measurement geometry is the opposite of ASTM standards E2194 and E2539, which define two specular illuminations at 15° and 45° and six specular spectral channels at 0°, 30°, 65°, -20°, -30°, and -60° for measuring metallic and pearlescent pigments. An additional illumination device 22 at a -45° angle is provided for use in conjunction with the pickup device 31 to measure gloss.

[0228] Two spectrometers, 31a and 32a, perform spectral analysis on the measurement light fed to them at observation angles of 45° and 15°, respectively, and each measurement produces a set of spectral measurements, each corresponding to an intensity within a different wavelength range. Spectrometers 31a and 32a do not perform spatial analysis on the measurement light; that is, they perform spectral analysis on the entire measurement light they receive.

[0229] The RGB camera 33a resolves both spatially and spectrally the measurement light fed to it at a 15° viewing angle. Based on the three colors RGB, the spectral resolution is limited to three channels. Each measurement by the RGB camera correspondingly produces a raw dataset of 3*n measurements, where n is the number of resolvable pixels.

[0230] An illumination device 28 provides diffuse illumination, enabling the measuring device to also support measurement modes with diffuse illumination conditions. The illumination device 28 is configured for LED background illumination, directly illuminating the object being measured from a large spatial angle. It includes two rows of white light-emitting diodes arranged on both sides of the measuring opening 6 and two inclined diffuse films, each assigned to one row for uniform illumination. The two rows of LEDs can be separately controlled by a control array.

[0231] Of course, depending on the intended application, other types of irradiation devices may also be used.

[0232] The aforementioned handheld measuring device is equipped with seven illumination devices and three pickup devices for measurement purposes. Other combinations of illumination devices and pickup devices are possible. The illumination devices do not necessarily need to be arranged in a plane.

[0233] The output of the measuring device includes a set of appearance attributes. Some of these appearance attributes are color attributes, such as RGB, CIELAB, or CIEXYZ values, either in the form of spectral measurements determined by a spectrometer or color values ​​derived from it. Other appearance attributes are texture attributes, such as texture data, either in the form of raw image data obtained from an RGB camera or in the form of image data derived from it, such as monochrome grayscale image data. This set of appearance attributes obtained from the measuring device is typically sparse compared to the appearance attributes required to generate an instance of the appearance model, as will be discussed in more detail in the next section.

[0234] Example of appearance model

[0235] As outlined above, many different appearance models have been proposed in the art. An example, particularly useful for describing the appearance of coatings on modern vehicles, will be described below. This exemplary appearance model is an improvement upon the model proposed in [RMS+08]. The contents of [RMS+08] are incorporated herein by reference in their entirety for the purpose of teaching the exemplary appearance model. The appearance model currently discussed is also referred to as the "CPA2 model".

[0236] The appearance model assumes the coating consists of two outermost layers: an opaque or translucent color layer, which is covered by a smooth, transparent coating. The color layer may contain highly reflective particles (so-called "flakes"). Incident light with direction i travels along its "refracted" incident direction. The light is refracted into the transparent coating before reaching the color layer. The reflected light travels along the outgoing direction before being refracted into the surrounding air in one direction (o). Propagation occurs through the transparent coating. Refraction is modeled using standard geometric optics, assuming the typical refractive index of the transparent coating.

[0237] The appearance model of the color layer is divided into three parts:

[0238] (a) Monochromatic luminance BRFD component, which models the spatial and wavelength-averaged overall reflection behavior (luminance variation) of the material. The luminance BRFD component can represent high frequencies in the angular domain.

[0239] (b) The spectral portion in the form of a discrete color table, which describes the low-frequency angular variations (color shifts) of the color.

[0240] (c) A BTF portion in the form of a texture table, which captures the effect of reflective flakes in the color layer. The BTF portion is represented by multiple fixed-size images, each representing the texture under a specific combination of illumination and viewing angles.

[0241] These three parts are combined into a BRDF function representing the spatial variation of the appearance of the painted surface:

[0242]

[0243] here,

[0244] i represents the direction of incident light in the air.

[0245] o represents the direction of reflected light in the air.

[0246] x is the position on the surface of the object.

[0247] n is the surface normal at position x.

[0248] The direction of (refraction) of incident light in the transparent coating.

[0249] The direction of (refractive) light reflected in the transparent coating.

[0250] F(n, d, η) is a function approximating Fresnel reflection at the boundary of a medium with a relative refractive index η (or the Fresnel formula itself).

[0251] For the monochrome BRDF model from (a),

[0252] For the color table interpolated from (b),

[0253] For the texture table of interpolation from (c),

[0254] δ(d1, d2) is the Dirac delta function on O(3), which is nonzero only when d1 = d2, and

[0255] r(n, d) is the direction of reflection d at the surface where the normal n is located.

[0256] (a) Example of the luminance BRDF section

[0257] A BRDF model based on multi-lobed micro-facets is used to model the monochromatic luminance BRDF component:

[0258]

[0259] here,

[0260] for and The direction along the middle (also known as the "middle vector")

[0261] a is the diffuse albedo.

[0262] K represents the number of BRDF lobes.

[0263] s k Let be the multiplier of the k-th lobe.

[0264] D is the normal distribution function of the micro-surface elements according to the selected model.

[0265] α k Let be the roughness parameter of the k-th lobe.

[0266] F 0，k Let be the Fresnel function parameter of the k-th lobe, which may be the refractive index or the exponent of reflection at perpendicular incidence.

[0267] G is the so-called geometric structure term based on the selected model.

[0268] In some embodiments, a Cook-Torrance BRDF model with a Schlicker-Fresnel approximation is used. Here, D, F, and G are defined as follows:

[0269]

[0270] F(n, d, F) 0，k ) = F 0，k +(1-F 0，k (1-)<n,d> ) 5

[0271]

[0272] here,

[0273] θ h Represents the mid-way vector in a fixed object reference frame. The polar angle, where the surface normal n defines the z-axis (“North Pole”).

[0274] These represent the vectors in the reference frame, respectively. and The z-component.

[0275] The free parameters of this model are a and s. k α k and F 0,k (For k = [1..K]). Typically, K is chosen to be between 1 and 3.

[0276] (b) Examples of the spectral section

[0277] The spectral portion is modeled using a discrete color table, which comprises values ​​of one or more color attributes (e.g., hue and saturation) for combinations of multiple directions. The values ​​of these color attributes will be simply referred to as "color values" below. Advantageously, the Rusinkiewicz parameterization [Rus98] is used to indicate the directions. In a simplified embodiment, the color table is bivariate, meaning the values ​​of the color attributes depend only on the polar angles (half-angles and difference angles) in the Rusinkiewicz parameterization, as will be explained below. These polar angles will also be referred to below as the "coordinates" of the color table.

[0278] Figure 6 An exemplary representation of a bivariate discrete color table is shown. On the horizontal axis, the half-angle θ h That is, the mid-way vector The polar angle is used as the first coordinate. The half-angle is also called the "flake angle" because it defines the angle between the surface normal n of the paint layer and the normal of the flake in the paint layer, which will cause incident light to change from the incident direction. Reflected in the direction of emission In the middle. On the vertical axis, the midway vector. and incident direction The range angle θ between I Used as the second coordinate. Range angle θ I Incident direction in the transformed reference frame The polar angle, in the transformed reference frame, the mid-way vector Define the z-axis. It can be interpreted as indicating the direction of incidence relative to the slice normal. The color table contains each pair of coordinates (i.e., the polar angle θ). h and θ I Each combination of coordinates represents one or more color values. The color table is defined for each pair of coordinates containing one or more color values. Figure 6 The circle in the diagram represents such a pair of coordinates (polar angle θ). h and θ I The combination of these (positions) will be referred to as “positions” in the color table below. Positions in the color table are spaced regularly. Therefore, the color table is represented by a regular rectangular grid of positions and their associated color values. The positions of two exemplary entries in the color table are labeled “a” and “b”.

[0279] The appearance model assumes that the paint material has isotropic reflection. For isotropic materials, color does not depend on the mid-range vector in a fixed object reference frame. azimuth φ h Therefore, the color table does not consider the azimuth angle φ. h .

[0280] Furthermore, the model assumes that the effect particles (“plates”) themselves also have isotropic reflection, and that the color shift is dominated by the specular reflection component of the reflected light. Therefore, the color depends only slightly on the incident direction in the transformed reference frame. azimuth φ I Therefore, the color chart does not consider the azimuth angle φ. I .

[0281] exist Figure 6 In the color table, at θ h and θ I The blank areas at larger values ​​indicate combinations of "forbidden" angles, which are physically impossible due to refraction and total reflection at the interface between the transparent coating and the surrounding air. The color chart does not include any color values ​​for these combinations of angles.

[0282] (c) Examples of BTF sections

[0283] The appearance model includes a bidirectional texture function (BTF) model. Generally, the BTF model is a multidimensional function dependent on the planar texture coordinates (x, y) and the viewing and illumination spherical angles. The BTF portion is modeled using a discrete texture table. Each entry in the texture table is a texture slice, representing the effect of a slice for a specific combination of illumination and viewing directions. In a simplified embodiment, Rusinkiewicz parameterization is also used for the texture table, and the texture table is again bivariate, again determined by the angle θ. h and θ I To parameterize.

[0284] Figure 7 A simplified example of a texture slice is shown in the figure. The bright spots in the figure indicate the reflection of the slice, whose slice normal is at the associated slice angle φ. h Orientation, and its position at the associated difference angle φ I It can be seen everywhere.

[0285] Figure 8 The discrete properties of the bivariate texture table are illustrated. In the appearance model, discrete positions are marked by circles (by their coordinates θ). h θ I (Definition) Provides texture slices. Due to the higher storage requirements per texture slice compared to color values ​​in a color table, θ in the texture table... h and θ I The spacing between the discrete values ​​may be larger than that in the color table. Similar to the color table, there are no entries for "forbidden" angles in the texture table. The positions of three exemplary texture slices are labeled a', b', and c'.

[0286] Instances of generating appearance models

[0287] The currently proposed methods, based on measurements using handheld appearance capture devices or from formulation software, are limited to only a limited number of directions. A limited set of appearance properties is available for combinations of light. These appearance properties are typically not available for combinations of the same incident and outgoing light directions required by the discrete color and texture tables in the appearance model. Furthermore, the available appearance properties are not necessarily of the same type as those required by the appearance model. For example, the available appearance properties typically do not include the parameters of the monochromatic luminance BRDF model as described above.

[0288] Therefore, to generate instances of the appearance model, the appearance attributes of the appearance model need to be determined from a finite set of available appearance attributes. It should be noted that this set of appearance attributes of the appearance model may have a much larger cardinality than the finite set of available appearance attributes. Therefore, generating instances of the appearance model may involve interpolation and extrapolation.

[0289] As discussed above, the free parameters of the luminance BRDF part are a and s. k α k and F 0,k (For k = [1..K]). These parameters can be determined using nonlinear optimizations known in the art [RMS+08].

[0290] Next, the bivariate color table needs to be populated. The color attributes may only apply to a few pairs of angles θ. h and θ I Available, in Figure 6 The diagram is marked with a cross. The color attribute corresponds to each pair of angles (θ) available to it. h ,θ I (This will be referred to as a "sample point" below.) A sample point defines the convex hull, which lies within... Figure 6 The area shown is represented by a dashed line. For this, each position in the bivariate color table requires a color value (θ). h ,θ I The locations will be referred to as "target locations". In this example, target locations are distributed on a regular grid, and therefore these locations can also be called "grid points". The color values ​​at the target locations will be called "target color values". They may have the same format as the available color attributes, or they may have different formats. For example, available color attributes may include spectral data, while color tables may include reduced values ​​(e.g., RGB tri-color values, CIEXYZ tri-color data, or values ​​in another color space such as CIELAB). For the following discussion, it is assumed that the transformation between the format of the available color attributes at the sample points and the format of the color values ​​in the color table is known.

[0291] Color values ​​at target locations can be easily interpolated and extrapolated from available color attributes at sample points using standard interpolation and extrapolation procedures. If the format of color values ​​in the color table differs from the format of available color attributes at sample points, necessary transformations can be applied before or after interpolation / extrapolation, provided that both the available color attributes at sample points and the color values ​​in the color table are expressed in a linear color space. If color values ​​in the color table are not expressed in a linear color space (e.g., as CIELAB values), interpolation / extrapolation is performed first in the linear color space, and then the transformations are applied.

[0292] For example, for interpolation within a convex hull, nearest neighbor interpolation, inverse distance weighting [Sh68], or Gaussian process regression [W98] can be used between color attributes at available sample points or between color values ​​obtained from these color attributes through the transformation, as is well known in the art.

[0293] In a preferred embodiment, for the minimum θ with a value less than the convex hull h θ h or higher than the maximum θ of the convex hull h θ h Extrapolating the target location outside the convex hull can be done using the color attribute (or derived color value) of the nearest neighbor sample point. The nearest neighbor sample point should be understood as having the smallest Euclidean distance from the target location. The sample points. Figure 6 This is illustrated for the target location "a". The color value at the target location "a" is derived from the available color attributes at sample point A, which is the nearest neighbor sample point to the target location "a".

[0294] Extrapolation at all other target locations outside the convex hull, especially with low or high θ. I For extrapolation at those locations, the preferred embodiment will not only use nearest neighbor sample points, but will also interpolate between the color attributes (or derived color values) of the two sample points closest to the target location. Figure 6 This is illustrated by the entry at target location "b". The two closest sample points are points B and C. Therefore, the color value at target location "b" will be determined by interpolation between the color attributes (or derived color values) at sample points B and C.

[0295] Of course, other “well-performing” interpolation and extrapolation methods can also be used, as they are known in the art.

[0296] Finally, the bivariate texture table needs to be populated. Texture samples in the form of images may only be for a few pairs of angles θ. h and θ I(Hereinafter referred to again as “sample points”) are available. For example, as described above, the appearance capture device 52 may have one or more cameras. The available texture samples may be image data for multiple illumination angles and a single viewing angle (e.g., in the case of a single RGB camera) or image datasets for multiple illumination angles and multiple viewing angles (e.g., in the case of multiple RGB cameras).

[0297] exist Figure 8 In the example, only six texture samples are available. The corresponding sample points are in Figure 8 The points are marked with a cross and labeled A', B', C', D', E', and F'. The convex hulls of these sample points are again shown as dashed lines. The texture samples at the sample points are called the "source textures." Texture slices need to be determined to fill the bivariate texture table with respect to the angle θ. h θ I This is again referred to as the "target location" with "target coordinates." Similar to the case of the color table, in this example, these points are arranged on a regular grid and therefore can also be called "grid points." Therefore, the texture slice at the target location is called the "target texture." They can be determined from the available source textures as follows:

[0298] -For each target location (θ) in the texture table h ,θ I This determines whether the target location is inside the convex hull.

[0299] - If the target location is inside the convex hull, then "Statistical Texture Composition" is used to interpolate the target texture at that location from the surrounding source textures, as explained in more detail below. For example, for Figure 8 Position a' in the text will be synthesized using statistical texture.

[0300] - If the target location is not within the convex hull, the target texture is defined using the source texture of the nearest neighbor sample point (“First Strategy”), or a constant texture without any spatial variation is used, i.e., an “empty” texture that does not contain any reflections from the slab (“Second Strategy”). In one possible implementation, for a maximum θ smaller than the available sample points... h The value θ h The first strategy can be adopted, and for the maximum θ greater than the available sample points... h θ h Adopt the second strategy. For example, for Figure 8 For position b', the first strategy would be appropriate, while for position c', the second strategy could be used. It is also possible to use statistical texture synthesis for target positions outside the convex hull, provided that at least two sample points are sufficiently close to the target position. For example, using the source textures at sample points B' and D', statistical texture synthesis for target position b' might be appropriate.

[0301] Statistical texture synthesis

[0302] Now refer to Figures 9 to 13 To explain Figure 8 An exemplary embodiment of statistical texture synthesis for target position a'.

[0303] Initially, three source textures at different sample points were selected for the target location a'. For this, a Delaunay triangulation was created for all sample points A' to F'. In two dimensions, as in this example, the result of the Delaunay triangulation is a set of triangles. Figure 8 In the diagram, the sides of the resulting triangle are plotted as straight lines from each sample point A' to F'. Now, select the triangle containing the target location a' and determine the sample points at the corners of this triangle. In this example, these are sample points B', D', and E'. For subsequent processes, the source texture at these three sample points will be used.

[0304] Next, the interpolation weights of the three source textures are determined. To do this, the absolute centroid coordinates of the target position a' relative to the selected triangle are determined, i.e., the non-negative centroid coordinates λ1, λ2, and λ3 satisfying the condition λ1 + λ2 + λ3 = 1. These centroid coordinates are then used as the interpolation weights of the source textures at the corners of the selected triangle. Figure 8 In the example, the absolute barycentric coordinates relative to sample point B' are quite large, while the absolute barycentric coordinates relative to sample points D' and E' are much smaller. Therefore, the source texture at sample point B' will receive a relatively large interpolation weight, while the interpolation weights at sample points D' and E' will be much smaller. For all other source textures, the interpolation weights are set to zero.

[0305] Now, synthesize the target texture for the target location a' from the three source textures at sample points B', D', and E'. This will be about Figure 9 To explain.

[0306] Therefore, one of three source textures is randomly selected, with the probability proportional to the interpolation weight of the source texture. Figure 9 In the example, source texture 211 at sample point B' has been randomly selected.

[0307] From the source texture 211, texture patches are now extracted at random locations within the source texture, i.e., a small portion of the source texture, indicated by small rectangles within the source texture. Figure 9 In the example, texture patch 212 is extracted. The extracted texture patch 212 is now modified such that one of its statistical properties at least approximately matches the corresponding average statistical property. In this example, the extracted texture patch 212 is modified such that its pixel value histogram approximately matches the average pixel value histogram 202. This will now be explained in more detail.

[0308] Figure 10 The diagram illustrates an exemplary pixel value histogram. A pixel value histogram contains multiple relative frequency values, one of which corresponds to each discrete pixel value in the image. For example, if the pixel value ranges from 0 to 255, then for each pixel value between 0 and 255, the pixel value histogram will include the relative frequency of that pixel value in the image. In some embodiments, pixel values ​​can be binned, each bin corresponding to a range of pixel values, and the pixel value histogram can accordingly include a reduced number of relative frequency values, each representing the relative frequency of a pixel value within a bin. Pixel value histograms can be used to evaluate the distribution of brightness in an image, including overall brightness and contrast. For example, Figure 10 The pixel value map will indicate that the image contains some bright spots or areas in front of a relatively dark background.

[0309] exist Figure 9 In the diagram, all pixel value histograms are shown in a very simplified manner, with only five bins for each pixel value histogram. The pixel value histogram of the source texture 211 from which texture patch 212 is extracted is shown as histogram 213. The average pixel value histogram is shown as histogram 202. This average histogram has been obtained as a weighted average of the pixel histograms 213, 222, and 232 of the three source textures 211, 221, and 231 at sample points B', D', and E', respectively, with the weights being the interpolation weights determined above. The pixel values ​​of texture patch 212 are now modified to obtain a pixel value histogram that more closely matches the average pixel value histogram 202. The modification is performed by applying a monotonically non-decreasing pointwise transformation to each pixel value in texture patch 212. Histogram matching algorithms for finding suitable transformations are well known in the field of digital image processing associated with brightness and contrast modifications. The modified texture sheet is illustrated as texture sheet 214, and the resulting pixel value histogram is illustrated as histogram 215. The resulting pixel value histogram 215 now closely matches the average pixel value histogram 202.

[0310] The modified texture patch 214 is now inserted into the target texture 201 so that it seamlessly fits the texture content already present in the target texture 201. For this purpose, the first modified texture patch 214 can simply be placed in a corner of the target texture 201. Each subsequent texture patch is inserted into the target texture using a technique called “MinCut” or “Graphcut”. See reference [Kw03]. In very simple terms, the seam that enhances the visual smoothness between existing pixels and the newly placed patch is calculated. The texture patch and the existing content are stitched together along this seam.

[0311] exist Figure 11The diagram illustrates such an insertion step. In this example, a modified texture piece 214 is inserted into a target texture 201 that already includes texture piece 203. The position 204 of the newly inserted piece 214 is selected such that it overlaps with the existing piece 203. A seam 205 is calculated in the overlapping area such that the existing pixels and the newly inserted pixels along the seam appear visually smooth, i.e., no visible boundary is created along the seam, as explained in detail in [Kw03]. The existing piece 203 and the newly inserted piece 214 are stitched together along this seam, with the remainder of each piece discarded. This results in a larger piece 206.

[0312] Now repeat this process frequently as needed to completely fill the target texture 201.

[0313] Flowchart for filling discrete texture tables

[0314] Figure 12 A flowchart illustrating an exemplary process for generating an appearance model is shown. In step 601, the model generation software receives a set of available appearance attributes, which may have been determined by an appearance capture device measuring the target object 50 or by formulation software determining the components of a candidate formulation 60 (including the proportions of the components in the candidate formulation 60). In step 602, the software performs a fitting of a luminance BRDF model to the available appearance attributes. In step 603, the software populates a discrete color table based on the available appearance attributes. In step 604, the software populates a discrete texture table based on the available appearance attributes. For this purpose, it determines the target texture that forms the entries in the discrete texture table.

[0315] Flowchart for statistical texture synthesis

[0316] Figure 13A flowchart illustrating an exemplary process for determining a target texture associated with a target coordinate set is shown. In step 701, the model generation software receives multiple source textures and their associated coordinates. In step 702, the software creates a Delaunay triangulation. In step 703, the software identifies a simplex containing the target coordinates within the Delaunay triangulation. In step 704, the software determines the centroid coordinates of the target coordinates relative to the identified simplex and stores these as interpolation weights. In step 705, the software randomly selects one of the source textures at a corner of the selected simplex with probability based on the interpolation weights of the source textures. In step 706, the software randomly extracts a texture patch from the selected source textures. In step 707, the software modifies the patch to match its pixel value histogram with the average pixel value histogram. In step 708, the software inserts the modified patch into the target texture so that it seamlessly fits the existing texture content in the target texture. Steps 705 to 708 are repeated until the target texture is completely filled. The completed target texture can then be used as an entry in a discrete texture table.

[0317] Calculate the texture table of composite materials

[0318] Statistical texture synthesis can also be employed when calculating entries for a discrete texture table of a composite material (which is a mixture of several components (e.g., several components of a candidate material, which is a mixture of components according to a formulation determined by formulation software)). In this case, source pieces can be randomly sampled from source textures associated with different components based on their probability of concentration in the mixture.

[0319] This is Figure 14 The diagram illustrates the process. In step 801, the model generation software receives at least two source textures and one formulation. Each source texture is associated with a component of the composite material. In step 802, the software randomly selects one of the source textures with a probability based on an interpolation weight reflecting the concentration of that component in the composite material. In step 803, the software randomly extracts a texture patch from the selected source texture. In step 804, the software modifies the patch to match its pixel value histogram with the average pixel value histogram. In step 805, the software inserts the modified patch into the target texture so that it seamlessly fits the existing texture content in the target texture. Steps 802 to 805 are repeated until the target texture is completely filled. The completed target texture can then be used as an entry in a discrete texture table.

[0320] Considering absorption and scattering in composite materials

[0321] In some cases, when calculating the texture of a material that is a mixture of several components, statistical texture synthesis operations provide a target texture that does not match the texture that would be obtained if measurements of the actual material were performed. In particular, the perceived average color and / or brightness may not be a perfect match.

[0322] This is at least partly due to the fact that, in practice, the source texture of individual components is often based on measurements of samples containing effect pigments in a transparent base material, while actual mixtures may also contain non-effect pigments (toners). The statistical texture synthesis process described above does not take into account the fact that non-effect pigments partially absorb light, thereby preventing some of the incident light from reaching the effect pigments and some of the reflected light from reaching the observer.

[0323] To correct this effect, two methods can be considered.

[0324] The first approach is a simple heuristic. In this approach, adjustments are made to the pixel values ​​in each target texture to match at least one statistical property of the target texture with a corresponding baseline property. The statistical property to be matched with the baseline property can, in particular, be average brightness and / or average color. Preferably, the average color value over the region of the target texture is matched with a baseline color value. The baseline color value can be obtained from various sources. For example, these baseline properties can be obtained from simple color measurements of the target or test subject (without image acquisition), or the baseline properties can have been calculated using a formulation engine that does not consider texture but is known to be very accurate in predicting color.

[0325] Preferably, matching is performed in a perceptual color space, particularly a perceptually uniform color space (which seeks to make color attributes perceptually uniform, i.e., the same spatial distance between two colors in the color space equals the same amount of perceptual color difference). An example of a perceptual color space is the well-known CIELAB color space. Furthermore, using a perceptual color space has proven to provide better matching results for measurements than if a non-perceptual color space such as CIEXYZ were used.

[0326] During the adjustment process, for a given geometry k and for each coordinate in the color space, the pixel values ​​in the corresponding target texture are adjusted in such a way that the average value (e.g., L*, a*, or b*) along the color coordinates on the texture region matches the corresponding reference value. This can be done by multiplying each pixel value by the ratio between the reference value and the average value calculated before adjustment. This process can be repeated for each geometry required to define the instance of the appearance model.

[0327] If a reference color is unavailable at a specific geometry to which the target texture adjustment is to be performed, the missing reference color can be calculated using an appearance model. Specifically, the BRDF model f(p; ω) can be performed on those geometries where the reference color is available. i ,ω o Fitting the parameter p of ). Then, in arbitrary geometry. The BRDF model is evaluated to predict the reference color for this geometry.

[0328] The second approach is driven by physics and is more complex than the simpler heuristic approach of the first. The second approach has the following prerequisites:

[0329] • The concentration of non-effect pigments (toners) in the material whose texture is to be predicted should be known.

[0330] • The optical parameters describing the scattering and absorption behavior of non-effect pigments should be known, such as the scattering coefficient σ. s and absorption coefficient σ a .

[0331] The formulation engine should be able to determine the combined scattering and absorption coefficients σ based on the concentration of non-effect pigments and their individual scattering and absorption coefficients. s σ a .

[0332] The formulation engine then performs optical simulations of the luminous flux within the material. This flux is typically described by a set of correlated differential equations, each describing the flux in a set of directions with respect to depth x within the paint layer:

[0333] F i (x)

[0334] Here, i is a set of indices for such directions (possibly upwards or downwards). These differential equations are coupled through the scattering effect of toner or paint layer boundaries.

[0335] During normal operation of the configuration engine, the user is interested in the upward luminous flux, which describes the boundary conditions of the incident light. This allows for the simulation measurement device to be used, and the simulated luminous flux can be compared with measurements from the actual device.

[0336] However, the solution to the differential equation with respect to the boundary conditions is also valid at any depth x. That is, for a given relative depth, by evaluating a set of downward directions i1, i2, ..., i n The solution can be used to calculate what the incident flux on the effect pigment will be. By calculating the integral in these directions, the attenuation A(x) of the incident light can be calculated. Assuming reciprocity, it can be assumed that the same attenuation applies to the light reflected by the effect pigment.

[0337] Therefore, if the depth x is known or can be estimated, it can be determined by multiplying each pixel value by A. 2 (x) is used to adjust the pixel value T of a given target texture along each coordinate in the color space. pFor example, in a simple embodiment, it can be assumed that the effect pigment is embedded in the material at an average depth x. For example, if the material is a paint coating, the average depth can be easily estimated based on the thickness of the paint coating. In a more complex embodiment, A can be calculated. 2 (x) distribution on the depth profile.

[0338] Two methods in Figure 14 The diagram is shown as step 804. In this step, the pixel values ​​in the target texture are adjusted to correct for absorption and scattering.

[0339] Combination of different appearance capture devices

[0340] In the workflow discussed above, the appearance attributes of three types of objects are determined through measurement: target objects, test objects, and reference objects already generated for various reference materials. These measurements are typically performed in different locations. For example, in the case of vehicle repair, measurements of the target object are performed in a body shop, measurements of the test object are typically performed by the paint supplier, and measurements of the reduced reference material are typically performed in a paint development laboratory.

[0341] Therefore, these measurements will typically not be performed by the same appearance capture device. If the same type of appearance capture device is used for measurements of all three types of objects, only inter-instrument calibration is required. However, different types of appearance capture devices, potentially with varying degrees of complexity, can often be used at different locations. For example, a car body repair shop might use a relatively simple handheld multi-angle spectrophotometer without imaging capabilities (such as X-Rite's MA-5 instrument) to measure the target object, while a paint supplier might use a more complex handheld multi-angle spectrophotometer with imaging capabilities (such as X-Rite's MA-T6 or MA-T12 instrument) to measure the test object, and a paint development laboratory might use a highly complex stationary appearance capture device (such as X-Rite's TAC7 instrument) to determine the properties of a reference material, featuring numerous illumination and observation directions and highly sophisticated imaging capabilities.

[0342] Therefore, the amount of available information associated with the three types of objects may differ. This is in Figure 14 The diagram in the middle shows, Figure 14The diagram schematically illustrates the variation of information content along three dimensions: the first dimension is the available information content of the target object, the second dimension is the available information content of the test object, and the third dimension is the available information content of the reference material. Available information content can vary along each of these dimensions between "L" ("low," color attributes are available only for a limited number of combinations of illumination and viewing directions, and no texture attributes are available) and "H" ("high," both color and texture attributes are available, each for a large number of combinations of illumination and viewing directions).

[0343] We will now discuss possible strategies for handling different levels of information content.

[0344] (a) The information content is high for the target object, test object, and reference material. Figure 14 (in the block "H / H / H")

[0345] Ideally, the information content is high for each of the three dimensions; that is, a large number of combinations of color attributes for the target material, the test material, and each reference material, as well as a considerable number of combinations of texture attributes in image form for the irradiation and observation directions, are available. In this case, sufficient information is directly available to visualize the appearance of the target material based on the measured appearance attributes of the target object and to generate a first instance of an appearance model for this purpose. Sufficient information is also available to determine suitable candidate formulations (for which a good match with the appearance of the target material is expected), to visualize the appearance of candidate formulations based on the available appearance attributes of the reference materials, and to generate a second instance of an appearance model for this purpose. Finally, sufficient information is also available to visualize the test material and to generate another instance of an appearance model for this purpose, and to evaluate whether the appearance of the test material actually matches the appearance of the target material by determining a suitable difference norm and / or by visually comparing the visualizations of the target material and the test material. The aforementioned processes of color interpolation and extrapolation, as well as statistical texture synthesis, can be used directly without modification to generate relevant instances of appearance models that include both color and texture.

[0346] (b) The information content is low for the target object, but high for the test subjects and reference materials. high ( Figure 14 (in the block "L / H / H")

[0347] In some embodiments, an appearance capture device with both spectrophotometric and imaging capabilities (i.e., high associated information content) is used to perform measurements on test and reference objects, while a simpler device with only spectrophotometric capabilities but no imaging capabilities (i.e., low associated information content) is used to perform measurements on the target object. The question then arises: how to visualize the material appearance of the target object in a realistic manner, even without the measured texture properties of that material?

[0348] In this scenario, at least two strategies are conceivable. In a simple strategy, the entire virtual object is simply visualized without textures. For this, both the first and second instances of the appearance model can be generated using an "empty" texture table—that is, a texture table whose entries are all identical and do not exhibit any spatial variations. Equivalently, an appearance model without textures can be used to visualize the virtual object.

[0349] However, this process may not produce a fairly realistic impression of the appearance of the target object and the materials of the candidate formulation, and therefore may make it difficult to determine whether these appearances actually match.

[0350] Therefore, in the preferred strategy, the texture of the target object is predicted based on known texture properties associated with the reference material. Once the test object has been produced, the texture prediction of the test object can be additionally based on the texture properties of the test object. In other words, texture information associated with the materials of the reference material and / or the test object is used to visualize the appearance of the target material. To this end, the first instance of the appearance model can be “edited” using texture information belonging to another instance of the appearance model; that is, the texture information associated with the target material is replaced or modified by texture information associated with a different material. A method for “editing” appearance properties in this manner is described in US20150032430A1, the contents of which are incorporated herein by reference in their entirety.

[0351] A similar strategy can be used if a limited number of texture properties are available for the target material, but these properties are insufficient to meaningfully generate the texture portion of a first instance of the appearance model, or contain less texture information than is available for the reference material and / or the test object. For example, the first appearance capture device could be a multi-angle spectrophotometer configured to determine color attributes and only one or a few global texture properties, such as a global roughness parameter or a global flare parameter. In contrast, the available texture properties for the reference material and / or the test object could include image data of several combinations of illumination and observation directions. In this case, the texture properties associated with the target object can be modified to include image data derived from the available image data of the reference material and / or the test object. The image data associated with the target object should, of course, be consistent with the available measured texture properties of the target object, for example, with measured global parameters such as roughness or flare parameters. This can be ensured by appropriately modifying the pixel values ​​in the available image data of the reference and / or test materials such that the global parameters calculated from the modified image data correspond respectively to the measured global parameters of the target object.

[0352] The method used to modify pixel values ​​can be very similar to the method described above in the context of statistical texture synthesis, where the texture patch is modified such that its pixel value histogram approximately matches the average pixel value histogram. Note that both the pixel value histogram and the global roughness or flicker parameter represent examples of statistical properties associated with the image data. Similar to the example of statistical texture synthesis above, in this example, the image is also modified such that one of its statistical properties (here: the global parameter calculated from the image data) approximately matches the measured statistical property of the target object (here: the measured global parameter of the target object).

[0353] The finite texture properties measured for the target object can also differ from the global parameters. For example, the measured texture properties may include a single image of a single combination of illumination and observation directions. Regardless of the exact nature of the measured texture properties, additional images of the target material can be generated by modifying the pixel values ​​of available images associated with the target material, reference material, and / or test object in such a way that at least one statistical property of the modified image data approximates the corresponding statistical property associated with the measured texture properties of the target object.

[0354] The proposed texture prediction process is based on the assumption that the texture of the target material will be quite similar to that of the candidate formulation, provided that the components of the candidate formulation belong to the same material category as the components of the target material. For example, if the target material is known to include a certain type of flake, and if the candidate formulation includes such a type of flake, it is reasonable to assume that the texture of the target material will be at least close to that of the candidate formulation.

[0355] A similar strategy can be employed if the available color attributes of the target material are insufficient to reliably fit the color attributes required by the appearance model used. This may be the case, for example, if the multi-angle spectrophotometer used does not have a sufficient number of illumination and observation direction pairs to determine the color attributes of the target material. In this case, the color attributes of the appearance model can also be determined by “editing” in a manner similar to the editing process for the texture attributes described above, using a procedure disclosed in US20150032430A1.

[0356] (c) The information content is high for the target object and reference materials, but low for the test object. of( Figure 14 (in the block "H / L / H")

[0357] In some embodiments, image-based texture information may be available for both the target and reference materials, but no texture information or only limited texture information (e.g., one or more global texture parameters or image information for only one pair or for the illumination and observation directions) may be available for the test object. In this case, a strategy similar to that in case (b) may be employed. In particular, the texture of the test material may be predicted based on known texture properties associated with the target material and / or the reference material.

[0358] (d) The information content is low for the target object and low for the test object, but low for... The reference material is highly informative. Figure 14 (in the block "L / L / H")

[0359] In some embodiments, image-based texture information is available only for the reference material, while no texture information or only limited texture information is available for the target and test objects. In this case, a strategy similar to that in case (b) can be employed again. Specifically, the texture of the target and test materials can be predicted based on known texture properties associated with the reference material.

[0360] (e) The information content is high for the target and test subjects, but low for the reference materials. ("H / H / L", in) Figure 14 (Not visible in the middle)

[0361] In some embodiments, image-based texture information may be available for both the target and test objects, but no texture information or only limited texture information may be available for the reference material. This may be the case, for example, if a relatively old colorant database is used, which was filled when only less sophisticated instruments were available, while more modern appearance capture devices are available to measure both the target and test materials.

[0362] In such a case, the texture information of the target material and / or test material (once the test material has been produced) can be used to visualize the appearance of the candidate formulation. For this purpose, a second instance of the associated appearance model can be “edited” using the texture information belonging to the target material and / or test material. This can be done in very the same way as the first instance of editing the appearance model in case (b) above.

[0363] Once these texture properties are available, it is advantageous to use the texture properties of the test material to visualize the candidate formulations because it can be safely assumed that the relatively small differences between the composition of the other candidate formulations and the test material will have only a negligible impact on the texture.

[0364] (f) The information content is low for the target object and reference materials, but high for the test object. ("L / H / L", in) Figure 14 (Not visible in the middle)

[0365] If image-based texture information is available for the test material, but no texture information or only limited texture information is available for the target and reference materials, the virtual object can be visualized without texture or using some generic texture, as long as the test object is not yet available. Once the test object is available, its texture properties can then be used to visualize the target material and candidate formulations. For this purpose, the texture properties in the associated instances of the appearance model can be edited as described above for case (b).

[0366] (g) The information content is high for the target object, but low for the test object and reference material. of( Figure 14 (in the block "H / L / L")

[0367] If image-based texture information is available for the target material, but no texture information or only limited texture information is available for the test and reference materials, a strategy similar to that for case (e) above can be employed. The texture information of the target object can be used to visualize the appearance of the candidate formulation and the test object. For this purpose, the texture information belonging to the target material can be used to “edit” associated instances of the appearance model.

[0368] (h) The information content is low for the target object, test object, and reference material. Figure 14 (in the block "L / L / L")

[0369] If no image-based texture information is available for any object in the object, the virtual object can be simply visualized without texture or using a generic texture of the material category to which the target object belongs.

[0370] References

[0371] [Ber19] Roy S. Berns, “Billmeyer and Saltzman’s Principles of Color Technology”, 4th ed., Wiley, 2019, p. 184.

[0372] [Kw03]Kwatra,V., A., Essa, I., Turk, G., and Bobick, A., “Graphcuttextures: image and video synthesis using graph cuts”, ACM Transactions on Graphics (ToG), 2003, 22(3), pp. 277-286.

[0373] [MTG+19] Gero Müller, Jochen Tautges, Alexander Gress, Martin Rump, Max Hermann and Francis Lamy, “AxF – Appearance exchange Format”, version 1.7, April 11, 2019, is available upon request from X-Rite.

[0374] [RMS+08]Martin Rump, Gero Müller, Ralf Sarlette, Dirk Koch and Richard Klein, "Photo-realistic Rendering of Metallic Car Paint from Image-Based Measurements", in: R. Scopigno, E. (Edited), Computer Graphics Forum, Eurographics, April 2008, 27:2 (527-536).

[0375] [RSK09] Martin Rump, Ralf Sarlette, and Richard Klein, “Efficient Resampling, Compression and Rendering of Metallic and Pearlescent Paint”, in: M. Magnus, B. Rosenhahn, H. Theisel (eds.), Proceedings of Vision, Modeling, and Visualization, November 2009, pp. 11-18.

[0376] [Rus98] Szymon M. Rusinkiewicz, “A New Change of Variables for Efficient BRDF Representation,” in: G. Drettakis G., N. Max (eds.), Rendering Techniques'98, Eurographics, Springer, Vienna, 1998. https: / / doi.org / 10.1007 / 978-3-7091-6453-2_ 2 .

[0377] [Sh68] Shepard D., "A two-dimensional interpolation function forirregularly-spaced data", in: Proceedings of the 196823rd ACM nationalconference, 1968, pp. 517-524.

[0378] [W98] Williams, CK, "Prediction with Gaussian processes: From linearregression to linear prediction and beyond", in: Learning in graphical model, Springer, Dordrecht, 1998, pp. 599-621.

[0379] List of reference markers

[0380] 1, 2 Grip components

[0381] 3 Wristbands

[0382] 4 Display Array

[0383] 5. Shell base

[0384] 6. Measure the opening

[0385] 7 substrate

[0386] 7a, 7b, 7c Supporting components

[0387] 10. Curved body

[0388] Irradiation devices 21, 22, 23, 24, 25, 26, 27, 28

[0389] Pickup devices 31, 32, and 33

[0390] 31a and 32a spectrometers

[0391] 33a RGB camera

[0392] 31c and 32c optical fibers

[0393] 50 Target Objects

[0394] 52. Appearance capture device

[0395] Appearance attributes 54 and 64

[0396] Examples of appearance models 56 and 66

[0397] 60 candidate formulations

[0398] 70 monitors

[0399] 72 Virtual Objects

[0400] Parts 72a, 72b, and 72c

[0401] Virtual separator lines 74 and 76

[0402] 80 test subjects

[0403] 82 Appearance capture device

[0404] 90 pointing device

[0405] 102 Model Generation Software

[0406] 104 Configuration Software

[0407] 106 Database (Colorants and Formulations)

[0408] 108 rendering software

[0409] 110 Database (Geometric Data)

[0410] 201 Target Texture

[0411] 211, 221, 231 Source Material

[0412] 203, 206, 212, 214 texture films

[0413] Histogram of pixel values: 202, 213, 215, 222, 232

[0414] 204 Location

[0415] 205 seam

[0416] 300 client computers

[0417] 310 processor

[0418] 320 Non-volatile Memory

[0419] 321 Operating System Software

[0420] 330 RAM

[0421] 340 I / O interfaces

[0422] 350 communication interface

[0423] 360 Server Computer

[0424] Steps 501-511

[0425] HMD handheld measuring device

[0426] H shell

[0427] DN device normal

[0428] SP System Plane

[0429] Sample points A, B, and C (color)

[0430] a, b Target locations (color chart)

[0431] Sample points (texture) A', B', C', D', E', F'

[0432] a', b', c' Target locations (texture table)

Claims

1. A computer-implemented method for visualizing the appearance of at least two materials, the method comprising: Obtain a first set of appearance attributes, the appearance attributes in the first set being associated with the target material, the first set including measured appearance attributes (54) of the target material, the measured appearance attributes (54) having been determined based on measurements of a target object (50) including the target material; Determine the formulation of candidate materials composed of multiple components, so that the candidate materials have the expected appearance properties that match the appearance properties of the target material; Obtain a second set of appearance attributes, which are associated with the candidate material. The second set includes appearance attributes that have been calculated from predetermined appearance attributes associated with multiple reference materials that include the candidate material. Obtain a geometric model of at least one virtual object (72), the geometric model defining the three-dimensional macroscopic surface geometry of the virtual object (72); and The scene including the at least one virtual object (72) is visualized using a display device (70), using first and second sets of appearance attributes and the geometric model, using the first set of appearance attributes to visualize the first part (72a) of the at least one virtual object (72), and using the second set of appearance attributes to visualize the second part (72b) of the at least one virtual object (72). Each of the first and second groups of appearance attributes includes a texture attribute in the form of image data. Its features are, The image data in the first group is calculated based on one or more texture properties associated with the reference material and / or with the candidate material, or The image data in the second group were calculated based on the texture properties associated with the target material and / or with test subjects (80) including previously identified different candidate materials.

2. The method according to claim 1, wherein, The image data in the first group is calculated based on image data associated with reference materials.

3. The method according to claim 1, wherein, The image data in the first group is based on the measured image data associated with the test subject (80).

4. The method according to any one of claims 1-3, wherein, Calculating the image data in the first group involves modifying the pixel values ​​of the images associated with the candidate material to match at least one statistical property of the first group with the known statistical properties of the target material.

5. The method according to claim 4, wherein, The known statistical properties of the target material are global texture attributes, particularly global roughness or glitter parameters.

6. The method according to claim 1, wherein, The image data in the second group is based on the texture properties associated with the target material and / or the test object (80).

7. The method according to claim 6, wherein, The image data in the second group is based on measured image data associated with the target material.

8. The method according to claim 6, wherein, The image data in the second group is based on measured image data associated with the test subjects.

9. The method according to any one of claims 6-8, wherein, Calculating the image data in the second group involves modifying the pixel values ​​of images associated with the target material and / or the test subject to match at least one statistical property of the second group with the known statistical properties of the candidate material.

10. The method according to claim 9, wherein, The known statistical properties of the candidate materials are global texture attributes, particularly global roughness or glitter parameters.

11. A computer-implemented method for visualizing the appearance of at least two materials, the method comprising: Obtain a first set of appearance attributes, the appearance attributes in the first set being associated with the target material, the first set including measured appearance attributes (54) of the target material, the measured appearance attributes (54) having been determined based on measurements of a target object (50) including the target material; Obtain the second set of appearance attributes, which are associated with the candidate materials; Obtain a geometric model of at least one virtual object (72), the geometric model defining the three-dimensional macroscopic surface geometry of the virtual object (72); and The scene including the at least one virtual object (72) is visualized using a display device (70), using first and second sets of appearance attributes and the geometric model, using the first set of appearance attributes to visualize the first part (72a) of the at least one virtual object (72), and using the second set of appearance attributes to visualize the second part (72b) of the at least one virtual object (72). Each of the first and second groups of appearance attributes includes a texture attribute in the form of image data. Its features are, Candidate materials are composite materials comprising at least two components according to a formulation that defines the concentration of each component in the composite material. The second set of appearance attributes includes at least one target texture in the form of image data, the target texture indicating the spatial variation of the composite material's appearance, and The method includes generating a target texture (201) using multiple source textures (211, 221, 231), each source texture (211, 221, 231) being associated with one of the components. The generation of the target texture (201) includes: (i) Assign interpolation weights to each of the source textures (211, 221, 231); and (ii) Synthesize the target texture (201) using the source texture (211, 221, 231) and the assigned interpolation weights. Among them, the target texture (201) for synthesis includes: (a) Randomly select one of the source textures (211, 221, 231) with a probability proportional to the interpolation weight of the source texture (211, 221, 231); (b) Randomly extract texture patches (212) from the selected source texture (211); (c) Modify the extracted texture patch (212) by modifying the pixel values ​​in the extracted texture patch to obtain a modified texture patch (214), wherein the pixel value modification is performed in such a way that at least one statistical property (215) of the modified texture patch (214) approximates the corresponding average statistical property (202), and the average statistical property (202) is determined by performing a weighted average on the source textures (211, 221, 231) by interpolation weights; (d) Insert the modified texture patch (214) into the target texture such that the modified texture patch (214) seamlessly fits the existing texture content (203) in the target texture; and (e) Repeat steps (a)-(d) until the target texture is completely filled.

12. The computer-implemented method of claim 11, comprising adjusting pixel values ​​in a target texture to correct for absorption and scattering effects in a composite material.

13. The computer-implemented method according to claim 12, in, Pixel values ​​are adjusted in such a way that at least one statistical property of the target texture matches the baseline properties of the composite material after adjustment.

14. The computer-implemented method according to claim 13, in, The pixel values ​​are adjusted in such a way that the average color space value of the target texture matches the reference color space value of the composite material after adjustment, wherein the color space value and the reference color space value are preferably expressed in a perceptual color space.

15. The computer-implemented method according to claim 14, in, The target texture (201) indicates the spatial variation of the composite material's appearance with respect to a specific combination of illumination and viewing directions, and The method includes fitting the parameters of a BRDF model to a reference color under multiple other combinations of illumination and viewing directions, and evaluating the BRDF model under the specific combination to obtain a reference color space value for that specific combination.

16. The computer-implemented method according to claim 12, comprising: To obtain individual optical parameters that at least approximately describe the scattering and absorption behavior of each component in the composite material; The combined optical parameters describing the scattering and absorption behavior of composite materials are determined based on the concentration of the components and their individual optical parameters. Optical simulations of the luminous flux within the composite material are performed for at least one layer beneath the surface of the composite material to determine the attenuation factors of incident and reflected light of the effect pigments in the layer. as well as The pixel values ​​of the target texture are adjusted based on the attenuation factor.

17. A computer-implemented method for visualizing the appearance of at least two materials, the method comprising: Obtain a first set of appearance attributes, the appearance attributes in the first set being associated with the target material, the first set including measured appearance attributes (54) of the target material, the measured appearance attributes (54) having been determined based on measurements of a target object (50) including the target material; Obtain the second set of appearance attributes, which are associated with the candidate materials; Obtain a geometric model of at least one virtual object (72), the geometric model defining the three-dimensional macroscopic surface geometry of the virtual object (72); and The scene including the at least one virtual object (72) is visualized using a display device (70), using first and second sets of appearance attributes and the geometric model, using the first set of appearance attributes to visualize the first part (72a) of the at least one virtual object (72), and using the second set of appearance attributes to visualize the second part (72b) of the at least one virtual object (72). Each of the first and second groups of appearance attributes includes a texture attribute in the form of image data. The method is characterized by comprising: Generate the first instance of the appearance model (56), the first instance of the appearance model including the first set of appearance attributes; and A second instance of the appearance model is generated (66), which includes a second set of appearance attributes. The appearance model includes a discrete texture table comprising multiple target textures (201), each target texture (201) represented by image data and associated with a different target coordinate set (a'), which indicates a specific combination of illumination and viewing directions. The first and second instances of the appearance model are used to visualize the at least one virtual object (72).

18. The computer-implemented method according to claim 17, wherein, All target textures (201) in the discrete texture table are dissimilar to each other on a pixel-by-pixel basis. If the pixel values ​​of two textures are statistically uncorrelated, they are considered dissimilar on a pixel-by-pixel basis.

19. The computer-implemented method according to claim 17 or 18, in, Generating the first and / or second instance (56, 66) of the appearance model includes at least one of the following operations: Interpolation is performed between available appearance attributes under different combinations of illumination and observation directions; as well as Extrapolate the available appearance properties from the combination of the selected illumination and observation directions.

20. The computer-implemented method according to claim 19, in, Available appearance attributes include multiple source textures (211, 221, 231), each source texture being associated with a different source coordinate set (A' to F'), each source coordinate set (A' to F') indicating a combination of illumination and viewing directions, and The first and / or second instances (56, 66) of generating the appearance model include determining at least one of the target textures (201) by performing statistical texture synthesis, the statistical texture synthesis including: (i) Assign interpolation weights to each of the source textures (211, 221, 231) based on the target coordinate set (a') and the source coordinate sets (A' to F'); and (ii) Synthesize the target texture (201) using the source texture (211, 221, 231) and the assigned interpolation weights. Among them, the target texture (201) for synthesis includes: (a) Randomly select one of the source textures (211, 221, 231) with a probability proportional to the interpolation weight of the source texture (211, 221, 231); (b) Randomly extract texture patches (212) from the selected source texture (211); (c) Modify the extracted texture patch (212) by modifying the pixel values ​​in the extracted texture patch to obtain a modified texture patch (214), wherein the pixel value modification is performed in such a way that at least one statistical property (215) of the modified texture patch (214) approximates the corresponding average statistical property (202), wherein the average statistical property (202) is determined by performing a weighted average on the source textures (211, 221, 231) by interpolation weights; (d) Insert the modified texture patch (214) into the target texture such that the modified texture patch (214) seamlessly fits the existing texture content (203) in the target texture; and (e) Repeat steps (a)-(d) until the target texture is completely filled.

21. The computer-implemented method according to any one of claims 1-3, 6-8, or 11-18, wherein, The texture attribute in each of the first and second groups of appearance attributes comprises multiple image datasets, each associated with a different combination of illumination and viewing orientation.

22. The computer-implemented method according to any one of claims 1-3, 6-8, or 11-18, comprising: The appearance capture device (52) is used to perform measurements on the target object (50) to determine multiple measured image datasets of the target material, each of which is associated with a different combination of illumination and observation directions.

23. The computer-implemented method according to any one of claims 1-3, 6-8, or 11-18, further comprising: The measured appearance properties of the test object (80), including the candidate material, are determined by performing measurements on the test object (80) using the appearance capture device (82). And it also includes at least one of the following steps: Visualize at least a portion of the at least one virtual object (72) using the measured appearance attributes of the test subject (80); and / or The modified formulation was determined using the measured appearance properties of the test subject (80) and the calculated appearance properties of the candidate materials.

24. An apparatus for visualizing the appearance of at least two materials, comprising a display device, at least one processor, and at least one memory, said at least one memory including program instructions configured to cause said at least one processor to perform the method described in any of the preceding claims.

25. A computer program product comprising program instructions, wherein the program instructions, when executed by at least one processor, cause the at least one processor to perform the method according to any one of claims 1 to 23.

Images