System and method for matching color and appearance of target coating

Abstract

The invention relates to a system and method for matching the color and appearance of a target coating. Systems and methods include receiving target image data associated with a target coating. Feature extraction analysis processing is applied to the target image data to determine target image features. The feature extraction analysis processing comprises the step of dividing the target image into sub-images containing a plurality of target pixels. The machine learning model uses the target pixel features to identify one or more types of slices present in the target coating.

Description

[0001] This application is a divisional application of the application filed on December 31, 2020, with application number 202011636892.6 and entitled "System and method for matching the color and appearance of a target coating". Cross-reference to related applications

[0002] This application claims priority to U.S. Provisional Patent Application No. 62 / 955,732, filed December 31, 2019, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The technical field relates to coating technology, and more specifically, to systems and methods for matching the color and appearance of a target coating. Background Technology

[0004] The visualization and selection of coatings with desired colors and appearances play a crucial role in many applications. For example, paint suppliers must offer thousands of paints to cover the range of coatings used by global OEMs for all current and latest vehicle models. Offering such a large number of different paints as factory-packaged products increases the complexity of paint manufacturing and inventory costs. Therefore, paint suppliers provide mixing systems for paint formulations that typically include 50 to 100 components (e.g., single pigment colors, binders, solvents, additives) and components that match the range of coatings available for the vehicle. The mixing system can be located at a repair shop (i.e., a body shop) or paint dealership and allows users to obtain a coating with the desired color and appearance by dispensing the components in amounts corresponding to the paint formulation. Paint formulations are typically maintained in a database and distributed to customers via computer software through download or direct connection to an internet database. Each paint formulation typically involves one or more alternative paint formulations to account for coating variations due to changes in vehicle production.

[0005] Identifying the paint formulation most similar to the target coating is complicated by this variation. For example, a particular coating might appear on three vehicle models manufactured in two assembly plants with different application equipment using paints from two different OEM paint suppliers and exceeding five years of vehicle life. These sources of variation result in significant coating variations within the group of vehicles with that particular coating. Alternative paint formulations provided by paint suppliers are matched to a subset of the color group, ensuring a close match for any vehicle requiring repair. Each alternative paint formulation can be represented by a color chart in a bar chart, allowing users to select the best matching formulation through visual comparison with the vehicle.

[0006] Identifying the paint formulation most similar to the target coating to be repaired is typically accomplished using a spectrophotometer or a fandeck. The spectrophotometer measures one or more color and appearance attributes of the target coating to be repaired. This color and appearance data is then compared to corresponding data from possible candidate formulations included in a database. The candidate formulation whose color and appearance attributes best match those of the target coating to be repaired is then selected as the paint formulation most similar to the target coating. However, spectrophotometers are expensive and not readily available in the market.

[0007] Alternatively, a bar chart includes multiple sample coatings on pages or sheets within the chart. The sample coatings on the bar chart are then visually compared to the target coating being repaired. The formulation associated with the sample coating that best matches the color and appearance properties of the target coating is then selected as the paint formulation most similar to the target coating. However, bar charts are cumbersome to use and difficult to maintain because they require a large number of sample coatings to account for all coatings on vehicles on today's roads.

[0008] Therefore, it is desirable to provide systems and methods for matching the color and appearance of a target coating. Furthermore, other desirable features and characteristics will become apparent from the following summary and detailed description, the appended claims, and in conjunction with the accompanying drawings and background information. Summary of the Invention

[0009] This document discloses various non-limiting embodiments of a system for matching the color and appearance of a target coating, and various non-limiting embodiments of a method for matching the color and appearance of a target coating. In one non-limiting embodiment, the system includes, but is not limited to, a storage device for storing instructions and one or more data processors. One or more data processors are configured to receive a target image of the target coating. The target image includes target image data. Feature extraction analysis processing is applied to the target image data to determine target image features. The feature extraction analysis processing includes dividing the target image into sub-images containing multiple target pixels. The sub-images may include a single patch or not. Target pixel features of the sub-images are determined. A machine learning model is applied to identify one or more types of patches present in the target coating using the determined target pixel features.

[0010] A system and method may further include receiving target image data associated with a target coating. Feature extraction analysis processing is applied to the target image data to determine target image features. The feature extraction analysis processing includes dividing the target image into sub-images containing multiple target pixels. A machine learning model uses the target pixel features to identify one or more types of patches present in the target coating. Attached Figure Description

[0011] Other advantages of the disclosed subject matter will be readily apparent, as will be better understood by referring to the following specific embodiments in conjunction with the accompanying drawings, in which: Figure 1 This is a perspective view illustrating a non-limiting embodiment of a system for matching the color and appearance of a target coating; Figure 2 It is shown Figure 1 A block diagram of a non-limiting implementation of the system; Figure 3A It is shown Figure 1 Images of non-limiting embodiments of the target coating; Figure 3B It is shown Figure 3A A graphical representation of the RGB values ​​of a non-limiting embodiment of the target coating; Figure 4A It is shown Figure 1 Images of a non-limiting implementation of the system's first sample image; Figure 4B It is shown Figure 4A A graphical representation of the RGB values ​​of the first sample image in a non-limiting implementation; Figure 5A It is shown Figure 1 Images of a non-limiting embodiment of the system's second sample image; Figure 5B It is shown Figure 5A A graphical representation of the RGB values ​​of the second sample image in a non-limiting embodiment; Figure 6 It is shown Figure 1 A perspective view of a non-limiting embodiment of the electronic imaging device of the system; Figure 7 It is shown Figure 1 Another perspective view of a non-limiting embodiment of the electronic imaging device of the system; Figure 8 It is shown Figure 1 A flowchart of a non-limiting implementation of the system; Figure 9 It is shown Figure 8 A flowchart of a non-limiting implementation of the method; Figure 10 It is shown Figure 8 A flowchart of another non-limiting implementation of the method.

[0012] Figure 11 This is a schematic diagram showing the formation of a cross-sectional portion of the substrate and coating; Figure 12 and Figure 13This is a schematic diagram showing an image of the captured coating; Figure 14 and Figure 15 This is a flowchart illustrating implementation methods of the system and method; Figure 16 This is a hypothetical diagram illustrating one possible part of a technique used to determine the characteristics of a target and / or sample; and Figures 17 to 19 This is a flowchart illustrating an implementation of the method.

[0013] Figure 20 This is a schematic diagram showing the application of repair coating to a substrate.

[0014] Figures 21 to 23 A color matching analysis system for analyzing one or more features of a target coating is described.

[0015] Figure 24 A machine learning model was described that was applied to the feature distribution and statistical data of the target coating.

[0016] Figure 25 A color analysis expert system that applies color analysis expert rules to predict the output chip probability is described.

[0017] Figure 26 A system for assisting shaders in understanding and correcting differences in spatial colors is described.

[0018] Figure 27 The processing flow for determining when a shader achieves an acceptable match is described.

[0019] Figure 28 A system for analyzing target coatings for texture matching is described.

[0020] Figure 29 It describes a server environment in which users can interact with a color and appearance matching analysis system. Detailed Implementation

[0021] The following detailed description includes examples and is not intended to limit the invention or its application and use. Furthermore, it is not intended to be limited by any theory set forth in the foregoing background or the following detailed description. It should be understood that throughout the drawings, corresponding reference numerals indicate the same or corresponding parts and features.

[0022] By reading the following detailed description, those skilled in the art will more readily understand the features and advantages identified in this disclosure. It should be understood that, for clarity, certain features described above and below in the context of different embodiments may also be provided in combination in a single embodiment. Conversely, for brevity, various features described in the context of a single embodiment may also be provided individually or in any sub-combination. Furthermore, unless the context clearly indicates otherwise, singular references may also include plurals (e.g., "a" and "an" may refer to one or more).

[0023] Unless otherwise expressly indicated, just as the minimum and maximum values ​​within a specified range are expressed using the word "approximately," the use of numerical values ​​within the various ranges specified in this disclosure is prescribed as approximations. In this way, substantially the same results as values ​​within the stated range can be obtained using small variations above and below the stated range. Moreover, these ranges are intended to be disclosed as continuous ranges including each value between the minimum and maximum values.

[0024] In this document, the process and technology can be described from the perspective of functional and / or logical block components and with reference to the symbolic representations of operations, processing tasks, and functions that can be performed by various computing components or devices. It should be understood that the various block components shown in the figures can be implemented by any number of hardware, software, and / or firmware components configured to perform specified functions. For example, the implementation of a system or component can employ various integrated circuit components capable of performing various functions under the control of one or more microprocessors or other control devices, such as memory elements, digital signal processing elements, logic elements, lookup tables, etc.

[0025] The following description may refer to elements, nodes, or features that are “coupled” together. As used herein, unless explicitly stated otherwise, “coupled” means that one element / node / feature is directly or indirectly engaged to (or directly or indirectly communicating with) another element / node / feature, and not necessarily mechanically engaged. Therefore, although the accompanying drawings may depict one example of the arrangement of elements, additional intervening elements, means, features, or components may be present in embodiments of the described subject matter. Additionally, certain terminology may be used in the following description for reference only and is therefore not intended to be limiting.

[0026] In this document, the process and technology can be described from the perspective of functional and / or logical block components and with reference to symbolic representations of operations, processing tasks, and functions that can be performed by various computing components or devices. Such operations, tasks, and functions are sometimes referred to as computer-executed, computerized, software-implemented, or computer-controlled. In practice, one or more processor devices can perform the described operations, tasks, and functions by manipulating electrical signals and other signal processing that represent data bits stored in the system memory. The storage location where the data bits are held is a physical location having specific electrical, magnetic, optical, or organic properties corresponding to these data bits. It should be understood that the various block components shown in the figures can be implemented by any number of hardware, software, and / or firmware components configured to perform specified functions. For example, the implementation of the system or components can employ various integrated circuit components that can perform various functions under the control of one or more microprocessors or other control devices, such as memory elements, digital signal processing elements, logic elements, lookup tables, etc.

[0027] For the sake of brevity, conventional techniques relating to graphics and image processing, touchscreen displays, and other functional aspects of certain systems and subsystems (and their independent operating components) may not be described in detail herein. Furthermore, the connecting lines shown in the various figures included herein are intended to illustrate examples of functional relationships and / or physical couplings between various elements. It should be noted that many alternative or additional functional relationships or physical connections may exist in embodiments of this subject matter.

[0028] As used herein, the term “module” means any hardware, software, firmware, electronic control components, processing logic and / or processor device, individually or in any combination, including but not limited to: application-specific integrated circuits (ASICs), electronic circuits, (shared, dedicated or grouped) processors and memory that executes one or more software or firmware programs, combined logic circuits and / or other suitable components that provide the described functionality.

[0029] As used herein, the term "pigment" or "multiple pigments" refers to one or more colorants that produce one or more colors. Pigments can be derived from natural or synthetic sources and are made from organic or inorganic components. Pigments may also comprise metal particles or flakes having a specific or mixed shape and size. Pigments are generally insoluble in coating compositions.

[0030] The term "effect pigment" or "multi-effect pigment" refers to pigments that produce special effects in a coating. Examples of effect pigments include, but are not limited to, light-scattering pigments, light-interference pigments, and light-reflecting pigments. Metallic flakes, such as aluminum flakes, and pearlescent pigments, such as mica-based pigments, are examples of effect pigments.

[0031] The term “appearance” can include: (1) aspects of the visual experience of observing or identifying a coating; and (2) the perception of combining the spectral and geometric aspects of the coating with its illumination and observation environment. Generally, appearance includes, in particular, the texture, grain, glitter, or other visual effects of a coating when viewed from different viewing angles and / or under different illumination conditions. Appearance characteristics or appearance data may include, but are not limited to, descriptive or measurement data regarding texture, metallic effect, pearlescent effect, gloss, image sharpness, sheet appearance, and size (e.g., texture, grain, glitter, luminescence, and shimmer, and the enhancement of depth perception in a coating imparted by the sheet, particularly by a metal sheet such as aluminum). Appearance characteristics can be obtained through visual inspection or by using an appearance measurement device.

[0032] The term "color data" or "color characteristics" for coatings can include measured color data, such as spectral reflectance values, C, U, and Z values. , , value, , , Color data includes values ​​such as L, C, h, or combinations thereof. Color data may also include the vehicle's color code, color name, or description, or combinations thereof. Color data may even include visual aspects of the coating's color, chromaticity, hue, brightness, or darkness. Color data can be obtained through visual inspection or by using a color measuring device such as a colorimeter, spectrophotometer, or goniometric spectrophotometer. Specifically, a spectrophotometer obtains color data by determining the wavelength of light reflected by the coating. Color data may also include: descriptive data, such as the color name or vehicle color code; binary, textured, or encrypted data files containing descriptive data for one or more colors; measurement data files, such as measurement data files generated by a color measuring device; or export / import data files generated by a computing device or color measuring device. Color data can also be generated by an appearance measuring device or a color-appearance dual measuring device.

[0033] The term "coating" or "coating composition" can include any coating composition known to those skilled in the art, and can include: two-component coating compositions, also known as "2K coating compositions"; one-component or 1K coating compositions; coating compositions having a crosslinkable component and a crosslinking component; radiation-curable coating compositions, such as UV-curable coating compositions or E-beam-curable coating compositions; single-curing coating compositions; dual-curing coating compositions; lacquer coating compositions; waterborne coating compositions or aqueous coating compositions; solvent-based coating compositions; or any other coating compositions known to those skilled in the art. Coating compositions can be formulated as primers, base coats, or colored coating compositions by incorporating desired pigments or effect pigments. Coating compositions can also be formulated as transparent coating compositions.

[0034] The terms “vehicle,” “automobile,” “motor vehicle,” or “motor vehicle” can include motor vehicles such as cars, buses, trucks, semi-trailer trucks, light cargo trucks, SUVs (sports utility vehicles); tractors; motorcycles; trailers; ATVs (all-terrain vehicles); heavy machinery such as bulldozers, mobile cranes, and excavators; aircraft; ships; vessels; and other modes of transport.

[0035] The terms "formulation," "matching formulation," or "matching scheme" used for coating compositions refer to a collection of information or instructions upon which the coating composition can be prepared. In one example, a matching formulation includes a list of the names and amounts of the pigments, effect pigments, and other components of the coating composition. In another example, a matching formulation includes instructions on how to mix the various components of the coating composition.

[0036] This article refers to Figure 1 A system 10 is provided, implemented by a processor, for matching the color and appearance of a target coating 12. The target coating 12 may be on a substrate 14. The substrate 14 may be a vehicle or part of a vehicle. The substrate 14 may also be any coated article including the target coating 12. The target coating 12 may include a colored coating, a clear coating, or a combination of colored and clear coatings. The colored coating may be formed from a colored paint composition. The clear coating may be formed from a clear paint composition. The target coating 12 may be formed from one or more solvent-based paint compositions, one or more water-based paint compositions, one or more two-component paint compositions, or one or more one-component paint compositions. The target coating 12 may also be formed from one or more paint compositions, each having a crosslinkable component and a crosslinking component, one or more radiation-curable paint compositions, or one or more varnish compositions.

[0037] Reference Figure 2 And continue to refer to Figure 1 System 10 includes an electronic imaging device 16 configured to generate target image data 18 for the target coating 12. The electronic imaging device 16 may be a device capable of capturing images over a wide range of electromagnetic wavelengths, including visible and invisible wavelengths. The electronic imaging device 16 may also be defined as a mobile device. Examples of mobile devices include, but are not limited to, mobile phones (e.g., smartphones), mobile computers (e.g., tablets or laptops), wearable devices (e.g., smartwatches or headsets), or any other type of device known in the art configured to receive the target image data 18. In one embodiment, the mobile device is a smartphone or tablet.

[0038] In one embodiment, the electronic imaging device 16 includes a camera 20 (see [link]). Figure 7 Camera 20 can be configured to acquire target image data 18. Camera 20 can be configured to capture images with visible wavelengths. Target image data 18 can be derived from a target image 58 of the target coating 12, such as a still image or video. In some embodiments, target image data 18 is derived from a still image. Figure 1 In the illustrated embodiment, the electron imaging device 16 is shown positioned near and spaced apart from the target coating 12. However, it should be understood that the electron imaging device 16 of this embodiment is portable, allowing it to be moved to another coating (not shown). In other embodiments (not shown), the electron imaging device 16 may be fixed in one location. In still other embodiments (not shown), the electron imaging device 16 may be attached to a robotic arm for automated movement. In yet another embodiment (not shown), the electron imaging device 16 may be configured to simultaneously measure the properties of multiple surfaces.

[0039] System 10 also includes a storage device 22 for storing instructions for performing color and appearance matching of the target coating 12. Storage device 22 may store instructions executable by one or more data processors 24. The instructions stored in storage device 22 may include one or more individual programs, each including an ordered list of executable instructions for implementing logical functions. When system 10 is operational, one or more data processors 24 are configured to: execute the instructions stored in storage device 22 to transfer data to and from storage device 22, and to generally control the operation of system 10 according to the instructions. In some embodiments, storage device 22 is associated with (or alternatively included in) devices such as: an electronic imaging device 16, a server associated with system 10, a cloud computing environment associated with system 10, or a combination thereof.

[0040] As described above, system 10 also includes one or more data processors 24 configured to execute instructions. One or more data processors 24 are configured to be communicatively coupled to the electronic imaging device 16. The one or more data processors 24 can be any custom or commercially available processor, central processing unit (CPU), auxiliary processor, semiconductor-based microprocessor (in the form of a microchip or chipset), or any general device for executing instructions among several processors associated with the electronic imaging device 16. The one or more data processors 24 can be communicatively coupled to any component of system 10 via wired connections, wireless connections, and / or devices or combinations thereof. Examples of suitable wired connections include, but are not limited to, hardware couplings, splitters, connectors, cables, or wires. Examples of suitable wireless connectivity and devices include, but are not limited to: Wi-Fi devices, Bluetooth devices, wide area network (WAN) wireless devices, Wi-Max devices, local area network (LAN) devices, 3G broadband devices, infrared communication devices, optical data transmission devices, radio transmitters and optional receivers, cordless phones, cordless phone adapter cards, or any other device capable of transmitting signals at a wide range of electromagnetic wavelengths, including radio frequency, microwave frequencies, visible wavelengths, or invisible wavelengths.

[0041] Reference Figure 3A and Figure 3B One or more data processors 24 are configured to execute instructions to receive target image data 18 of the target coating 12. As described above, the target image data 18 is generated by the electronic imaging device 16. The target image data 18 can define RGB values ​​representing the target coating 12. Values ​​or combinations thereof. In some embodiments, target image data 18 defines RGB values ​​representing target coating 12. One or more data processors 24 may also be configured to execute instructions to convert the RGB values ​​of target image data 18 into values ​​representing target coating 12. value.

[0042] Target image data 18 includes target image features 26. Target image features 26 may include the color and appearance characteristics of the target coating 12, a representation of the target image data 18, or a combination thereof. In some embodiments, target image features 26 may include an image entropy-based representation.

[0043] One or more data processors 24 are configured to execute instructions to retrieve one or more feature extraction analysis processes 28' for extracting target image features 26 from target image data 18. In an embodiment, one or more feature extraction analysis processes 28' are configured to: identify an image entropy-based representation to extract target image features 26 from target image data 18. To this end, one or more data processors 24 may be configured to execute instructions to identify an image entropy-based representation to extract target image features 26 from target image data 18.

[0044] Recognizing an image entropy-based representation may include: determining the color image entropy curve of the target image data 18. This can be based on each plane, each plane, each Shannon entropy of a plane or a combination thereof is obtained using color entropy curves in three dimensions. The target image data 18 is represented in space. Determining the color entropy curve may include: [determining the three-dimensional representation of the target image data 18]. The space is divided into multiple cubic subspaces; cubic spaces with similar properties are tabulated to obtain the total cubic space count for each property; a three-dimensional representation is generated. An empty image entropy array for each dimension of the space; and an empty image entropy array filled with the total cubic space count corresponding to each dimension.

[0045] Recognition based on image entropy representation may also include: determining the color difference image entropy curve of the target image data 18. This can be achieved using three-dimensional information about the alternative. Three-dimensional spatial analysis Space in three dimensions The target image data is represented in space 18. Determining the chromatic entropy curve may include: calculating three-dimensional... Three-dimensional space and alternatives Between spaces Image entropy, Image entropy and Image entropy.

[0046] Recognition based on image entropy may also include: determining the three dimensions from the target image data 18. spatial The black-and-white intensity image entropy of a plane. Recognition based on image entropy representations may also include: determining the average value of the target image data 18. Value. Recognizing image entropy-based representations may also include: determining the center of the densest cubic subspace. value.

[0047] One or more data processors 24 are further configured to execute the instructions described above to apply the target image data 18 to one or more feature extraction and analysis processes 28'. One or more data processors 24 are further configured to execute the instructions described above to extract target image features 26 from the target image data 18 using one or more feature extraction and analysis processes 28'.

[0048] In one implementation, system 10 is configured to extract target image features 26 from target image data 18 by recognizing an image entropy-based representation of target image features 26. Recognizing the image entropy-based representation may include: determining the color image entropy curve of the target image data 18; determining the color image entropy curve of the target image data 18; and determining the three-dimensional representation from the target image data 18. spatial The entropy of a black-and-white intensity image on a plane; determining the average of the target image data 18. Value; determines the center of the densest cubic subspace. Value; or a combination thereof.

[0049] Reference Figure 4A and Figure 5A And continue to refer to Figure 2 In this embodiment, system 10 also includes a sample database 30. The sample database 30 may be associated with or separate from the electronic imaging device 16, for example, in a server-based or cloud computing environment. It should be understood that one or more data processors 24 are configured to be communicatively coupled to the sample database 30. The sample database 30 may include a plurality of sample images 32, for example... Figure 4A The first sample image 34 shown is Figure 5A The second sample image 36 is shown. In this embodiment, each of the plurality of sample images 32 is an image of a panel including a sample coating. Various sample coatings defining a set of coating formulations can be imaged to generate the plurality of sample images 32. The sample images 32 can be imaged using one or more different electronic imaging devices 16 to account for variations in the imaging capabilities and performance of each electronic imaging device 16. The plurality of sample images 32 can be in any format, such as RAW, JPEG, TIFF, BMP, GIF, PNG, etc.

[0050] One or more data processors 24 may be configured to execute instructions to receive sample image data 38 of sample image 32. Sample image data 38 may be generated by electronic imaging device 16. Sample image data 38 may define RGB values ​​representing sample image 32. Values ​​or combinations thereof. In some implementations, sample image data 38 defines RGB values ​​representing sample image 32, for example... Figure 4B The RGB values ​​of the first sample image 34 shown are Figure 5B The second sample image 36 is shown as having RGB values. One or more data processors 24 may also be configured to execute instructions to convert the RGB values ​​of the sample image data 38 into a representation of the sample image 32. The system 10 can be configured to normalize sample image data 38 of multiple sample images 32 for various electronic imaging devices 16, thereby improving the performance of the system 10.

[0051] Sample image data 38 may include sample image features 40. Sample image features 40 may include color and appearance characteristics of sample image 32, a representation of sample image data 38, or a combination thereof. In some embodiments, sample image features 40 may include a representation based on image entropy.

[0052] One or more data processors 24 are configured to execute instructions to retrieve one or more feature extraction analysis processes 28” for extracting sample image features 40 from sample image data 38. In an embodiment, one or more feature extraction analysis processes 28” are configured to: identify an image entropy-based representation to extract sample image features 40 from sample image data 38. To this end, one or more data processors 24 may be configured to execute instructions to identify an image entropy-based representation to extract sample image features 40 from sample image data 38. It should be understood that one or more feature extraction analysis processes 28” for extracting sample image features 40 may be the same as or different from one or more feature extraction analysis processes 28' for extracting target image features 26.

[0053] In one implementation, system 10 is configured to extract sample image features 40 from sample image data 38 by recognizing an image entropy-based representation of the sample image features 40. Recognizing the image entropy-based representation may include: determining a color image entropy curve of the sample image data 38; determining a three-dimensional representation from the sample image data 38. spatial The entropy of a black-and-white intensity image in a plane; determining the average of 38 sample image data. Value; determines the center of the densest cubic subspace. Value; or a combination thereof.

[0054] One or more data processors 24 are configured to execute instructions to retrieve a machine learning model 42 from a plurality of sample images 32, the machine learning model 42 utilizing target image features 26 to identify computed matching sample images 44. The machine learning model 42 may employ supervised training, unsupervised training, or a combination thereof. In this embodiment, the machine learning model 42 employs supervised training. Examples of suitable machine learning models include, but are not limited to: linear regression, decision trees, k-means clustering, principal component analysis (PCA), random decision forests, neural networks, or any other type of machine learning algorithm known in the art. In this embodiment, the machine learning model is based on a random decision forest algorithm.

[0055] The machine learning model 42 includes a pre-specified matching criterion 46 representing the plurality of sample images 32 for identifying a computed matching sample image 44 from the plurality of sample images 32. In an embodiment, the pre-specified matching criterion 46 is arranged in one or more decision trees. One or more data processors 24 are configured to apply target image features 26 to the machine learning model 42. In an embodiment, the pre-specified matching criterion 46 is included in one or more decision trees, wherein the decision tree includes a root node, intermediate nodes running through different levels, and end nodes. The target image features 26 can be processed through nodes of one or more end nodes, wherein each end node represents one of the plurality of sample images 32.

[0056] One or more data processors 24 are further configured to identify computed matching sample images 44 based on substantially satisfying one or more pre-specified matching criteria 46. In an embodiment, the phrase "substantially satisfying" means identifying the computed matching sample image 44 from a plurality of sample images 32 with the highest probability of matching the target coating 12. In an embodiment, the machine learning model 42 is based on a random decision forest algorithm comprising multiple decision trees, wherein the result of processing the target image features 26 by each decision tree is used to determine the probability that each sample image 32 matches the target coating 12. The sample image 32 with the highest probability of matching the target coating 12 can be defined as the computed matching sample image 44.

[0057] In some implementations, one or more data processors 24 are configured to execute instructions to generate a pre-specified matching criterion 46 for a machine learning model 42 based on sample image features 40. In some implementations, the pre-specified matching criterion 46 is generated based on sample image features 40 extracted from a plurality of sample images 32. One or more data processors 24 may be configured to execute instructions to train a machine learning model 42 on a plurality of sample images 32 by generating a pre-specified matching criterion 46 based on sample image features 40. The machine learning model 42 may be trained on a plurality of sample images 32 included in a sample database 30 at regular intervals (e.g., monthly). As described above, data processors may be used to identify representations based on image entropy from a plurality of sample images 32. The sample image features 40 extracted from the sample image data 38, which defines the RGB values ​​of the sample image 32, are converted into... value.

[0058] The calculated matching sample image 44 is used to match the color and appearance of the target coating 12. The calculated matching sample image 44 may correspond to a paint formulation that may match the color and appearance of the target coating 12. System 10 may include one or more alternative matching sample images 48 associated with the calculated matching sample image 44. One or more alternative matching sample images 48 may be associated with the calculated matching sample image 44 based on the paint formulation, observed similarity, calculated similarity, or a combination thereof. In some embodiments, one or more alternative matching sample images 48 are associated with the calculated matching sample image 44 based on the paint formulation. In embodiments, the calculated matching sample image 44 corresponds to a primary paint formulation, and one or more alternative matching sample images 48 correspond to alternative paint formulations associated with the primary paint formulation. System 10 may include a visual matching sample image 50 that can be selected by a user from the calculated matching sample image 44 and one or more alternative matching sample images 48 based on the user's observed similarity to the target coating 12.

[0059] Reference Figure 6 and Figure 7 In some embodiments, the electronic imaging device 16 further includes a display 52 configured to display the calculated matching sample image 44. In some embodiments, the display 52 is also configured to display a target image 58 of the target coating 12 adjacent to the calculated matching sample image 44. In other embodiments, the display 52 is also configured to display one or more alternative matching sample images 48 related to the calculated matching sample image 44. In embodiments of the electronic imaging device 16 including the camera 20, the display 52 may be located opposite the camera 20.

[0060] In one embodiment, system 10 further includes a user input module 54 configured to allow a user to select a visual matching sample image 50 from a calculated matching sample image 44 and one or more alternative matching sample images 48 based on the user's observed similarity to the target coating 12. In an embodiment of the electronic imaging device 16 including a display 52, the user can select the visual matching sample image 50 by touch input on the display 52.

[0061] In one embodiment, system 10 further includes a light source 56 configured to illuminate the target coating 12. In an embodiment of an electronic imaging device 16 including camera 20, electronic imaging device 16 may include light source 56, and light source 56 may be positioned adjacent to camera 20.

[0062] In one embodiment, system 10 further includes a dark box (not shown) for isolating the target coating 12 to be imaged from external light, shadows, and reflections. The dark box may be configured to house the electronic imaging device 16 and allow the target coating 12 to be exposed to the camera 20 and the light source 56. The dark box may include a light diffuser (not shown) configured to cooperate with the light source 56 to adequately scatter the light generated from the light source 56.

[0063] This article also refers to Figure 8 And continue to refer to Figures 1 to 7 A method 1100 is provided for matching the color and appearance of a target coating 12. The method 1100 includes the following step 1102: receiving target image data 18 of the target coating 12 by one or more data processors. The target image data 18 is generated by an electro-imaging device 16 and includes target image features 26. The method 1100 further includes the following step 1104: retrieving one or more feature extraction analysis processes 28' from the target image data 18 by one or more processors to extract the target image features 26. The method 1100 further includes the following step 1106: applying the target image features 26 to one or more feature extraction analysis processes 28'. The method 1100 further includes the following step 1108: extracting the target image features 26 from the target image data 18 using one or more feature extraction analysis processes 28'.

[0064] Method 1100 further includes the step 1110 of retrieving a machine learning model 42 by one or more data processors, the machine learning model 42 identifying a computed matching sample image 44 from a plurality of sample images 32 using target image features 26. The machine learning model 42 includes pre-specified matching criteria 46 representing the plurality of sample images 32 for identifying the computed matching sample image 44 from the plurality of sample images 32. Method 1100 further includes the step 1112 of applying the target image features 26 to the machine learning model 42. Method 1100 further includes the step 1114 of identifying the computed matching sample image 44 based on substantially satisfying one or more pre-specified matching criteria 46.

[0065] In one embodiment, method 1100 further includes the step 1116 of displaying on display 52 a calculated matching sample image 44, one or more alternative matching sample images 48 associated with the calculated matching sample image 44, and a target image 58 of the target coating 12 adjacent to the calculated matching sample image 44 and one or more alternative matching sample images 48. In another embodiment, method 1100 further includes the step 1118 of selecting a visual matching sample image 50 from the calculated matching sample image 44 and one or more alternative matching sample images 48 by a user based on observed similarity to the target image data 18.

[0066] Reference Figure 9 And continue to refer to Figures 1 to 8 In an implementation, method 1100 further includes the following step 1120: generating a machine learning model 42 based on multiple sample images 32. Step 1120 of generating the machine learning model 42 may include the following step 1122: retrieving multiple sample images 32 from the sample database 30. Step 1120 of generating the machine learning model 42 may also include the following step 1124: extracting sample image features 40 from the multiple sample images 32 based on one or more feature extraction analysis processes 28'. Step 1120 of generating the machine learning model 42 may also include the following step 1126: generating a pre-specified matching criterion 46 based on the sample image features 40.

[0067] Reference Figure 10 And continue to refer to Figures 1 to 9 In an embodiment, method 1100 further includes the step 1128 of forming a coating composition corresponding to the calculated matching sample image 44. Method 1100 may also include the step 1130 of applying the coating composition to the substrate 14.

[0068] The method 1100 and system 10 disclosed herein can be used for any coated article or substrate 14 including the target coating 12. Some examples of such coated articles may include, but are not limited to: household appliances, such as refrigerators, washing machines, dishwashers, microwave ovens, cooking and baking ovens; electronic devices, such as televisions, computers, video game consoles, audio and video equipment; entertainment equipment, such as bicycles, ski equipment, all-terrain vehicles; and home or office furniture, such as desks, filing cabinets; watercraft or vessels, such as small boats, yachts or private watercraft (PWC); aircraft; buildings; structures, such as bridges; industrial equipment, such as cranes, heavy trucks or bulldozers; or decorative articles.

[0069] Color matching for effect pigment-based coatings is particularly challenging. Effect coatings include metallic and pearlescent coatings, but can also include other effects such as phosphorescence and fluorescence. Figure 11 As shown and continue to refer to Figures 1 to 10 The metallic coating and pearlescent coating include effect additive 74. The coating, including the target coating 12 and / or sample coating 60 as shown, may include several layers covering the substrate 14. The target coating 12 may include... Figure 11 The sample coating 60 shown has the same layers, therefore the description of the layers of sample coating 60 also applies to the target coating 12 described above. As used herein, the term "over" means that the intermediate layer can be located "over" between the upper cover (sample coating 60 in this example) and the lower cover (substrate 14 in this example), or means that the upper cover physically contacts the lower cover. Furthermore, the term "over" means that a vertical line passing through the upper cover also passes through the lower cover, such that at least a portion of the upper cover is directly above at least a portion of the lower cover. It should be understood that substrate 14 can be moved, causing the relative "up" and "down" positions to change. Spatial relative terms such as "top," "bottom," "above," and "below" are... Figure 11 This arises in the context of the orientation of the cross-section. It should be understood that spatial relative terms refer to... Figure 11 The orientation in the middle, therefore if the substrate 14 is oriented in another way, the spatial relative term will still refer to Figure 11 The orientation depicted in the figure. Therefore, even if the substrate 14 is twisted, flipped, or oriented in a manner different from that depicted in the figure, the terms "above" and "below" remain the same.

[0070] Figure 11A primer 62 and a base coat 64 are shown covering a substrate 14. In this specification, the primer 62 and substrate 14 are not considered part of the sample coating 60. An optional effect coating 66 covers the base coat 64, and a clear coat 68 covers the optional effect coating 66. The sample coating formulation 70 includes multiple components 72, wherein the component 72 for the base coat 64 may be different from the component 72 for the optional effect coating 66 and / or the clear coat 68. One or both of the base coat 64 and the effect coating 66 include an effect additive 74 as one of the components 72. The effect additive 74 is used to produce special effects on the sample coating 60, such as a metallic or pearlescent effect. The sample coating formulation 70 includes a base coat formulation 65 for the base coat 64, an optional effect coating formulation 67 for the optional effect coating 66, and a clear coating formulation 69 for the clear coat 68. In some embodiments, the sample coating formulation 70 may also include formulations for other optional layers.

[0071] A metallic effect is produced when the sample coating 60 (or any other coating) includes visible reflective sheets. The reflective sheets serve as effect additive 74. In this embodiment, metallic particles in the coating absorb and reflect incident light of more colors than the base coating, giving the coating a varied appearance over a given area. Some coatings will appear as the color of the base coat, while others will reflect light and appear as glitter or shimmer. Metallic color is the color that appears as a polished metallic color. The visual effect typically associated with metal is a metallic sheen, distinct from a simple solid color. Metallic color includes a luminous effect due to the brightness of the material, and this brightness varies with the angle of the surface relative to the light source. One technique for producing metallic effect colors is to add aluminum flakes (an example of effect additive 74) to the pigment coating. The aluminum flakes produce glitter varying in size, brightness, and sometimes color, depending on how the flakes are treated. Larger flakes produce coarser glitter, while smaller flakes produce finer glitter. Different types of flakes can be used, such as flat and relatively round “coin” flakes, like silver coins. Other types of flakes can have serrated edges, like corn chips. In some implementations, the sheet can also be colored, thus producing a colored flash.

[0072] Adding aluminum flakes to a base coat 64 produces a metallic effect, but if the same type and amount of flakes are added to a translucent effect coat 66 covering the base coat 64, the coating has a “deeper” and different appearance. In another embodiment, the base coat 64 may include one type and amount of effect additive 74, while the effect coat 66 may include different types and / or amounts of effect additive 74 to produce a different appearance. Therefore, many variables affect the appearance of a metallic color, such as the type of flake, the size of the flake, the coating including the flake, the base color, etc. Thus, it is difficult to match a metallic effect due to the various possible factors and appearances.

[0073] The pearlescent coating includes effect additives 74 that selectively reflect, absorb, and / or transmit visible light, producing a colored appearance that varies based on the structure and morphology of the flakes. This gives the coating a shimmering and dark color that varies with viewing angle and / or illumination angle. The effect additives 74 in the pearlescent coating can be ceramic crystals, and these effect additives 74 can be added to the base coat 64, the effect coat 66, or both. Furthermore, the pearlescent effect additives 74 can be in grades with variations in size, refractive index, shape, etc., and different grades can be used alone or in combination. The pearlescent effect additives 74 can also be combined with metallic effect additives 74 in various combinations, and the appearance of the coating will vary with changes in the concentration, type, position, etc., of the effect additives. All the different possible variations of the effect additives 74 can be applied to a single color, therefore, techniques that only match colors will be ineffective in reproducing the appearance of coatings based on effect pigments.

[0074] As mentioned above, coatings based on effect pigments have a varied appearance, therefore, such as Figure 3A , Figure 4A and Figure 5AAs illustrated in the diagram, different pixels within an image will have different colors, brightness, hues, chromaticity, or other appearance characteristics. Therefore, a color matching protocol that breaks down an image into pixels and analyzes it pixel by pixel to determine the pixel differences between two or more pixels can help match the overall appearance of a coating based on effect pigments. Image features can be determined using mathematical models, such as the machine learning models described above, based on the differences between pixels within an image. Thus, as described above, the target image 58 can be analyzed pixel by pixel using mathematical models to generate target image features 26, and this can be combined with other target image features 26, such as color (which can be determined by considering the target image 58 as a whole rather than pixel by pixel) or other target image features. Again, as described above, one or more target image features 26 obtained can be compared with a sample database 30 that has already generated similar sample image features 40 to find the best match. Pixel-by-pixel evaluation can produce a match for effect pigments that would not be possible with an evaluation based on considering the target image 58 as a whole.

[0075] refer to Figure 12 The implementation method shown is further referred to. Figures 1 to 11 As described above, the imaging device 16 captures a target image 58 of the target coating 12. During the capture of the target image 58, the imaging device 16 is positioned at an imaging angle 76 relative to the surface of the target coating 12, and the illumination source 78 is positioned at an illumination angle 80. The target image 58 is divided into a plurality of target pixels 82, wherein variations in the plurality of target pixels 82 cause at least one target pixel 82 to appear different from another target pixel 82.

[0076] In a similar manner, refer to Figure 13 The implementation method shown is further referred to. Figures 1 to 12 As described above again, the imaging device 16 captures a sample image 32 of the sample coating 60. The imaging device 16 used to capture the sample image 32 can be the same as the imaging device 16 used to capture the target image 58, but different imaging devices 16 can also be used. The illumination source 78 is also used to capture the sample image 32, wherein, similar to... Figure 12 As described with respect to target image 58, the imaging device is positioned at an imaging angle 76 relative to the surface of sample coating 60, while the illumination source 78 is positioned at an illumination angle 80. Sample image 32 is divided into a plurality of sample pixels 84, one of which has an appearance different from another. In an embodiment, sample coating 60 includes an effect additive 74, which is shown as a small dot in the illustration of sample image 32, and the effect additive 74 produces variations in the appearance of the sample pixels 84 within sample image 32. Figure 12 and Figure 13The target coating 12 and sample coating 60 may include a base coating 64, an optional effect coating 66, and a transparent coating 68, respectively, but variations in the coatings are also possible.

[0077] like Figure 14 The implementation method is shown in the figure and continues to refer to the following. Figures 1 to 13 This generates a sample database 30 for matching the target image 58 with the sample paint formulation 70. Generating the sample database 30 includes generating the sample paint formulation 70, which includes an effect additive 74. The sample paint formulation 70 may include a base coat 64 and a clear coat 68, or a base coat 64, an effect coat 66, and a clear coat 68, but other implementations are also possible. For example, any one of the base coat 64, the optional effect coat 66, and the clear coat 68 may include multiple layers, and other coatings may also be present.

[0078] Once a sample paint formulation 70 is produced, a sample coating 60 is generated using the sample paint formulation 70. This is typically done by applying material from the sample paint formulation 70 to the substrate 14, for example, via spraying, brushing, dip coating, digital printing, or any other coating technique. In one embodiment, the sample coating 60 is formed by spraying, wherein the spraying utilizes recommended spraying conditions for the paint grade in the sample paint formulation 70. Spraying conditions may include solvent type, solvent volume, spray gun pressure, type and / or size of the nozzle used in the spray gun, distance between the spray gun and the substrate 14, etc. Producing the sample coating 60 using the same techniques typically used in automotive body repair shops or other situations where matching the target coating 12 may be necessary provides a more accurate representation of the expected finished product compared to applying the sample coating 60 using a different technique. The sample paint formulation 70 includes one or more effect additives 74, so the sample coating 60 has a varied appearance, wherein one part of the sample coating 60 presents differently from another part. For example, a glittery finish presents a different color than a matte finish.

[0079] Then, an image 32 is generated using an imaging device 16, for example, by photographing the sample coating 60. This image 32 includes multiple image data 38, such as RGB values, Values, etc. Sample image 32 can be one or more still or moving images. In embodiments, sample image 32 includes multiple still images captured with known and specified illumination, imaging angle, distance between imaging device 16 and sample coating 60, and illumination angle. Sample image 32 may also target a substantially flat portion of sample coating 60, but in some embodiments, sample image 32 may also include one or more images of sample coating 60 with a known curvature.

[0080] Feature extraction and analysis processing 28 can be applied to sample image data 38 to generate sample image features 40. The sample image features 40 are then associated with sample coating formulations 70 and stored in a sample database 30. The sample database 30 may include multiple sample image features 40 associated with a sample coating formulation 70, and one or more sample images 32 may also be associated with a sample coating formulation 70. The sample database 30 is stored in one or more storage devices 22, which may be the same as or different from the storage device 22 described above for matching the target coating 12. As described above, the storage device 22 storing the sample database 30 can be associated with a data processor 24 to execute instructions for saving data and retrieving data from the sample database 30. This process can be repeated for multiple sample coating formulations 70. Figure 14 The processing shown thus stores multiple sample paint formulations 70 in the sample database 30. The sample paint formulations 70 can also be configured to nearly match the original equipment coating provided by the vehicle manufacturer, so that the vehicle coating matches the sample paint formulation 70. One or more sample images 32 can be used as the calculated matching sample images 44, or different sets of parameters can be used to capture the calculated matching sample images 44 from the sample coatings 60.

[0081] Multiple sample paint formulations 70 can be generated, producing nearly identical sample images 32, wherein the sample paint formulations 70 differ from one another and include different grades of paint. Some vehicle repair shops tend to use a certain grade of paint, and the grade of paint can vary from one vehicle repair shop to another. Because vehicle repair shops have the skill and familiarity to use a particular grade of paint, a sample database 30 that includes matching sample paint formulations 70 using the same grade of paint as the vehicle repair shop can improve results. In this way, different vehicle repair shops (or other users of the sample database 30) that typically use different grades of paint can utilize the same sample database 30.

[0082] exist Figure 15 In the implementation method and continue to refer to Figures 1 to 14 The above details are shown in more detail. Figure 14 The described feature extraction and analysis process 28” is used to divide the sample image 32 into multiple sample pixels 84. Each sample pixel 84 has sample pixel image data, and the sample pixel image data is used to determine the sample pixel features 86 of the sample pixel 84. Because the sample pixel 84 can have an appearance different from the overall sample image 32, the sample pixel features 86 can be different from the overall sample features 40. The sample pixel features 86 can be the RGB values ​​of the sample pixel 84 or the RGB values ​​of the sample pixel 84. The value or other appearance attributes of the sample pixel 84 are determined. In one embodiment, sample pixel features 86 are determined for all sample pixels 84 of the sample image 32, but in an alternative embodiment, sample pixel features 86 may be determined only for a subset of the sample pixels 84 of the sample image 32. In all embodiments, sample pixel features 86 are determined for a plurality of sample pixels 84, wherein the sample pixel features 86 vary for at least some of the sample pixels 84. Then, sample pixel feature differences 88 are determined for the sample pixels 84. Then, sample image features 40 are determined based on the sample pixel feature differences 88.

[0083] exist Figure 16 China and continue to refer to Figures 1 to 15 An implementation using a three-dimensional coordinate system 90 is illustrated. The three-dimensional coordinate system 90 can represent the RGB color system, wherein one axis of the three-dimensional coordinate system 90 is the R value, another axis is the G value, and a third axis is the B value. Alternatively, the three-dimensional coordinate system 90 can represent... Color system, in which one axis is The value, the other axis is The value and the third axis are Value. In alternative embodiments, the three-dimensional coordinate system 90 may represent other axes, and in some embodiments, the three-dimensional coordinate system 90 may not be used. Sample pixel features 86 can then be plotted in the three-dimensional coordinate system 90, and the number of sample pixels 84 falling within each block of the three-dimensional coordinate system 90 can be recorded. Figure 16 In the hypothetical illustration, the first block closest to the 0-0-0 coordinates has a count of 1, the block directly above the first block has a count of 2, and the block directly above the block with a count of 2 has a count of 5. Substitution techniques can be used to determine the sample pixel feature differences 88, and multiple techniques can be used to determine the multiple sample pixel feature differences 88 of a sample image 32 in multiple ways.

[0084] Then, as Figure 15 As shown and continue to refer to Figures 1 to 14 and Figure 16Sample image features 40 can be determined based on the differences 88 in sample pixel features. Since sample image features 40 are based on color or appearance variations in different spaces within the sample image 32, determining sample image features 40 based on multiple sample pixel feature differences 88 is referred to herein as spatial microcolor analysis. Multiple sample image features 40 can exist for a single sample image 32, and some of those sample image features 40 may be based on spatial microcolor analysis, while others may not. However, in this specification, at least one of the sample image features 40 is based on spatial microcolor analysis. Thus, the sample database 30 includes at least one sample image feature 40 based on spatial microcolor analysis. Figure 15 38 blocks of sample image data at the top and Figure 15 The sample image features at the bottom of the 40 blocks are shown in Figure 14 The implementation of the steps utilized between blocks with the same markings in the sample database 30 is described above. The calculated matching sample image 44 can also be stored in the sample database 30, whereby the calculated matching sample image 44 can be used to display to the user as described above. The user can then determine, based on visual analysis, whether the calculated matching sample image 44 is an acceptable match for the target coating 12. The calculated matching sample image 44 can be captured using the imaging device 16 if it is deemed acceptable for visual analysis.

[0085] refer to Figure 17 The implementation method shown is further referred to. Figures 1 to 16 . Figure 17 An implementation method for using spatial micro-color analysis on target image 58 is shown, such as... Figure 2 The feature extraction and analysis processing in block 28 is illustrated. Therefore, Figure 17 18 blocks of target image data at the top and from Figure 16 The third target image feature from the bottom is 26 blocks based on Figure 2 It was obtained from a block with the same name in the middle. Figure 17 The document also illustrates an implementation of a mathematical model 100 for comparing target image features 26 with sample image features 40, which is an implementation of a model other than machine learning model 42. However, it should be understood that machine learning model 42 may still be utilized in some implementations. As described above, Figure 2 and Figure 17 The sample image feature 40 shown is obtained from the sample database 30.

[0086] like Figure 12 and Figure 17As shown, the target image 58 is divided into a plurality of target pixels 82. Each target pixel 82 includes target pixel image data, wherein the target pixel image data is at least a portion of the target image data 18. As described above with respect to sample pixel features 86, target pixel features 102 are determined based on the target pixels 82, wherein each target pixel in the target pixels 82 has at least some target image data in the target image data 18, and said at least some target image data in the target image data 18 varies for different target pixels 82. Again, as described above with respect to sample pixel features 86, then target pixel feature differences 104 are determined based on target pixel features 102, and then target image features 26 are determined based on target pixel feature differences 104. Thus, because at least some of the target pixels 82 have different target image data 18, at least one of the target image features 26 is determined by spatial microcolor analysis. The target image features 26 of a single target image 58 may include one or more target image features 26 determined by spatial microcolor analysis, and may also include one or more target image features 26 not based on spatial microcolor analysis. Then, as described above, the calculated matching sample image 44 is determined using the pre-specified matching criteria 46.

[0087] Various spatial microcolor analysis mathematical techniques can be used to determine target image features 26 and / or sample image features 40, and in some implementations, the same techniques can be used to facilitate matching. A partial list of spatial microcolor analysis mathematical techniques is provided below, where the general term pixel refers to target pixel 82 or sample pixel 84, and image refers to target image 58 or sample image 32. Examples of spatial microcolor analysis mathematical techniques include, but are not limited to: determining the individual pixels of an image... Color coordinates; determine the average of each pixel in the entire image. Color coordinates; determining the flash region of a black and white image; determining the flash intensity of a black and white image; determining the flash level of a black and white image; determining the flash color of an image; determining the flash clustering of an image; determining the flash chromatic difference within an image; determining the flash persistence of an image, where flash persistence is a measure of flash as a function of one or more illumination changes during image capture; determining pixel-level color constancy under one or more illumination changes during image capture; determining wavelet coefficients of an image at the pixel level; determining Fourier coefficients of a target image at the target pixel level; determining the average color of a local region within an image, where the local region may be one or more pixels, but where the local region is smaller than the total area of ​​the image; determining the discrete color of an image. The pixel count within the range, where, The range can be fixed, or it can be data-driven to vary; determine the maximum fill coordinates of the pixel-level cubic bins of the image, where the cubic bins are based on using... Or 3D coordinate mapping of RGB values; determine the overall image color entropy; determine the third dimension of the image as a function of the color depth. Image entropy of one or more planes in the plane; determining image entropy of one or more planes in the RGB plane as a function of the third dimension of the image; determining a measure of local pixel variation of the image; determining the granularity of the image; determining the high variance vector of the image, wherein principal component analysis is used to establish the high variance vector; and determining the kurtosis vector of the image, wherein independent component analysis is used to establish the kurtosis vector.

[0088] Determine each pixel of the target / sample image Color coordinates involve breaking down the image into pixels and then determining the coordinates of multiple (or in some implementations, all) pixels. Color coordinates. Determine the average of each pixel in the entire image. Color coordinates refer to the values ​​taken for each pixel within the image. The mean of the sample. Determining the flash area of ​​a black and white image means acquiring or obtaining a black and white image, and then determining the number of pixels including the flash and the total number of pixels in a given region. The given region can be the entire image or a subset of the image. The flash area is then determined by dividing the number of pixels including the flash in the given region by the total number of pixels in that given region. Determining the flash intensity of a black and white image means determining the average brightness of the pixels including the flash within a given region.

[0089] Determining the flash level of a black-and-white image means determining the visually perceived value describing the flash phenomenon based on the flash area and flash intensity. Determining the flash color of an image means determining the location of the flash and then determining the color of that flash. Determining flash clustering of an image means using clustering or distribution fitting algorithms to determine the various colors of flashes present in the image. As mentioned above, determining flash chromatic aberration within an image means determining the color of flashes in the image and then determining the variations or differences in that color within the flash. Determining flash persistence of an image means determining whether the flash remains within a given pixel (and whether the flash has a change in brightness) when the illumination changes during image capture and the imaging angle 76 and illumination angle 80 remain the same. Determining pixel-level color constancy means determining whether the color of pixels within a given pixel remains the same when the illumination changes during image capture and the imaging angle 76 and illumination angle 80 remain the same. At least two different images are required to determine flash persistence and color constancy.

[0090] Determining the Fourier coefficients of a pixel-level image means determining the coefficients associated with the image after decomposing it into sinusoidal components of different frequencies using a Fourier transform, where the coefficients describe the frequencies present in the image. This can be determined for a given number of pixels within a given region of the image. Determining the wavelet coefficients of a pixel-level image means determining the coefficients associated with the image after decomposing it into components associated with shifted and scaled versions of the wavelet using a discrete or continuous wavelet transform, where the coefficients describe the image content associated with the relevant shifted and scaled versions of the wavelet. A wavelet is a function that tends to zero at extrema and contains oscillations in local regions. This can be determined for a given number of pixels within a given region of the image. Determining the average color of a local region within an image means determining the average color of one or more pixels within a sample region smaller than the total area of ​​the image. Determining the discrete... The pixel count within the range means determining the use The value is the pixel count within a given region of a three-dimensional coordinate system with axis 90, and... Figure 16 The illustration is similar. The size of a given region within a 90-degree 3D coordinate system can be fixed, or it can vary depending on the generated count values. Determine the maximum fill coordinates of the pixel-level cubic bins of the image—where the cubic bins are based on using... Or, using RGB values ​​as axes in a three-dimensional coordinate system, 90° – meaning as... Figure 16 The block shown is identified as having the highest pixel count. You can use a setting value, percentage value, or other metric to determine the value at which a block is designated to be filled with "maximum".

[0091] Determining the overall image color entropy means determining the overall color entropy of the image based on the changes within each pixel. Color entropy can be Shannon entropy or other types of entropy calculations. This is determined as a function of the third dimension of the image. Image entropy of one or more planes in the plane or RGB plane, meaning the selection of such planes... Figure 16The plane within the 3D coordinate system 90 shown is used, and then the entropy is determined based on the values ​​along this plane. As mentioned above, the entropy can be Shannon entropy or other types of entropy calculations. Determining the local pixel variation metric of the image means selecting essentially any pixel feature and then determining the variation of that pixel feature within the image. Determining the granularity of the image means determining the impression of the image granularity based on shading or other features that suggest granularity. Determining the high variance vector of the image means determining the vector based on the vector origin at the coordinate origin, where principal component analysis is used to establish the high variance vector. The high variance vector can be considered as target image feature 26 or sample image feature 40, or pixel data from the image can be projected onto the vector to obtain new target image feature 26 or sample image feature 40. Determining the kurtosis vector of the image also means determining the vector based on the vector origin at the coordinate origin, where independent component analysis is used to establish the kurtosis vector. The kurtosis vector can be considered as target image feature 26 or sample image feature 40, or pixel data from the image can be projected onto the vector to obtain new target image feature 26 or sample image feature 40.

[0092] like Figure 18 As shown and continue to refer to Figures 1 to 17 In another embodiment, a method for matching a target coating 12 is provided. The method includes obtaining a target image 58 of the target coating 1200, wherein the target coating 12 is a pigment-based coating including effect additives 74. The method also includes applying a feature extraction analysis process 28' to the target image 58, wherein the feature extraction analysis process 28' includes: dividing the target image 58 1220 into a plurality of target pixels 82 including target pixel image data; determining target pixel features 102 for each of the plurality of target pixels 82; determining target pixel feature differences 104 between the respective target pixels 82; determining target image features 26 based on the target pixel feature differences 104; and calculating a calculated matching sample image 44 using the target image features 26 based on substantially satisfying one or more pre-specified matching criteria.

[0093] like Figure 19 As shown and continue to refer to Figures 1 to 18The method also provides a method for generating a sample database 30. This method includes preparing 1300 sample coatings 60 according to a sample coating formulation 70 including an effect additive 74, such that the appearance of the sample coatings 60 varies from one location to another. The method also includes imaging the sample coatings 1310 to generate a sample image 32 including sample image data, wherein the sample image 32 is divided into a plurality of sample pixels 84, each sample pixel including sample pixel image data. A next step is to retrieve 1320 one or more sample image features 40 from the sample image data, wherein at least one of the sample image features 40 includes spatial microcolor analysis, which includes values ​​determined by sample pixel feature differences 88 between at least two sample pixels 84. Another step is to store 1330 the sample coating formulation 70 and one or more sample image features 40 in the sample database 30, wherein the sample coating formulation 70 is associated with one or more sample image features 40.

[0094] Reference Figure 20 Once the calculated matching sample image 44 is obtained, it can be approved by the operator. At this point, the repair coating 110 can be prepared using the sample coating formulation 70 corresponding to the calculated matching sample image 44. In an exemplary embodiment, the sample coating formulation 70 includes an effect additive 74 and one or more other components 72. The repair coating 110 can be applied to a substrate 14, such as a vehicle requiring repair, using one or more of a variety of techniques. In an exemplary embodiment, as shown, the repair coating 110 is applied to the substrate 12 using a digital printer 112 and digital printing technology; however, in alternative embodiments, the repair coating 110 can be applied to the substrate 12 using other techniques, including but not limited to spraying, brushing, and / or dipping.

[0095] Figure 21 A method for analyzing one or more features of a painted automotive part or the entire vehicle is described in section 1400. Examples of analysis types include determining the type of pigment flakes used on the painted object and using the analysis for matching purposes. The pigment flake selection process can be, for example, an automated part of a paint matching formulation development workflow for effect pigments. This contrasts with previous methods that used manual processing to identify possible effect pigments for matching purposes.

[0096] Image data 1404 of the painted object is provided to the analysis process. An electronic imaging device with inherent or supplementary illumination 1402 can generate image data 1404 of the target coating of the painted object for use in one or more analysis computer programs. The image data 1404 is placed at 1406 into a location such as... , , Within a specific color space, pre-specified features can be extracted. The extracted features can include any color or appearance features required for the matching process.

[0097] Feature extraction process 1408 parses the image data into regions of sub-images 1410. The degree of presence (or absence) of features within a sub-image is used to generate a distribution 1414 of features across sub-images 1410 at 1412. Furthermore, statistical measures are generated at 1412 to more completely describe the distribution 1414 of features within the image data 1404. For example, statistical measures used to describe the feature distribution 1414 may include standard deviation, mean, skewness measure, kurtosis measure, etc. Note that the spatial localization of color in metallic coatings cannot be captured solely by reflection.

[0098] A computer-implemented non-transient storage medium 1416 or multiple non-transient storage media 1416 stores the distribution and statistical data of features extracted from the representation. A machine learning model 1418 is applied to the stored data to predict the type of one or more patches present in the target coating. The machine learning model 1418 can employ supervised training, unsupervised training, or a combination thereof. Examples of suitable machine learning models include, but are not limited to: linear regression, decision trees, k-means clustering, principal component analysis (PCA), stochastic decision forests, neural networks, or any other type of machine learning algorithm known in the art. In this implementation, the machine learning model is based on a multilayer artificial neural network.

[0099] One or more machine learning models 1418 predict the types of flakes present in the target coating. The prediction output 1420 can represent the ratio of the most likely type of flakes to the least likely type of flakes present in the target coating. The formulation process can use the automatically determined flake types as part of a paint formulation for matching the target coating.

[0100] Figure 22 Described in 1500 Figure 21 Example configuration for processing. See [reference]. Figure 22 The illumination source 1402 can be an electronic imaging device for capturing images over a wide range of electromagnetic wavelengths, including visible and invisible wavelengths. The electronic imaging device may include one or more cameras 1502 of different types, such as cameras as part of a mobile device. Examples of mobile devices include, but are not limited to, mobile phones (e.g., smartphones), mobile computers (e.g., tablets or laptops), wearable devices (e.g., smartwatches or headsets), or any other type of device known in the art configured to receive target image data. The illumination of the coated target can be at different angles to capture different patch sizes and geometries.

[0101] After capturing image data 1402, the system can use multiple matrices (e.g., three matrices) to represent the color space. For example, the data structure for storing the three matrices 1504 can be configured as follows. In an implementation, three separate data structures can store... Value matrix, Value matrix and Each value in the value matrix. These values ​​represent each pixel in the entire captured image or a portion of the captured image. The values ​​stored in the matrix are converted from the RGB color space to... , , Color space. The RGB color space is device-dependent (e.g., different for each camera). Converting to... , , Color spaces make subsequent calculations independent of the device.

[0102] Based on color-related data stored in the data structure, feature extraction process 1408 selects which features to analyze to determine one or more patch types of the target coating. For example, the feature extraction process may extract features of sub-image 1410 at 1506, such as average... , , Color coordinates, flash intensity, and image size, etc.

[0103] Figure 23 An example configuration for further processing of the data contained in sub-image 1410 is depicted at 1600. Sub-image 1410 can be analyzed at 1602 to determine the distribution 1604 of one or more features across the sub-image. For example, the system can determine how many pixels within the sub-image include flashes associated with patches and the total number of such pixels. The system can further analyze the “size” feature associated with the patches to determine the distribution 1604 of patch sizes across sub-image 1410.

[0104] Figure 24 Example configuration 1700 depicts a machine learning model at 1702 that applies feature distributions and statistics stored in one or more color data storage media 1416 to generate a prediction probability 1704. The example configuration uses an artificial neural network 1706 to predict the slice type based on the input feature distributions and statistics.

[0105] The artificial neural network 1706 may consist of an input layer 1708, one or more hidden layers 1710, and an output layer 1712. The input layer 1708 may have one or more input nodes to receive feature distributions and statistical information related to the image from the target coating. One or more hidden layers 1710 within the artificial neural network 1706 help identify interrelationships within the input data. Because these interrelationships can be inherently non-linear, the artificial neural network 1706 is better able to identify patterns or desired objects (e.g., patches or pixels associated with patches) within a dataset from one or more color data storage media 1416. The nodes of the artificial neural network 1706 may use many different types of activation functions, such as sigmoid activation functions, logistic activation functions, etc.

[0106] The output layer 1712 of the artificial neural network 1706 provides a prediction at 1714 indicating the probability of the presence of a specific patch. The prediction output can represent the ratio of the most likely type of patch to the least likely type of patch present in the target coating. The prediction can also be provided as the first n patches.

[0107] In one implementation, the artificial neural network executing on one or more processors is a convolutional neural network (CNN), which is trained on a labeled image dataset to learn features of each type of object within the images. The labeled image dataset includes information identifying labeled sub-images and the category of each object. During training, the artificial neural model can predict categories based on features. Since the relationships between training input images and categories are predefined, the training module can adjust the model's predictions to match the predefined associations between input images and features. Once the model has been trained, a machine learning classifier can predict patch categories based on the data.

[0108] It should be understood that other machine learning models, such as random forest models and decision tree models, can be used. The machine learning models used within the systems disclosed herein include empirical machine learning models trained on data generated during real-world applications (e.g., color-related data generated by a spraying application).

[0109] It should also be understood that the predicted probabilities from the model's output can be further analyzed. Figure 25 The following example is provided for example, where a color analysis expert system 1804 applies its color analysis rule 1808 to the output prediction probability 1704 to determine whether one or more output candidate patches are more or less likely to exist in a sub-image of the target coating. For example, the color analysis expert system 1804 may determine that two patches are generally not present for the coating, and adjust the probability prediction at 1806 to reflect this actual level of detail.

[0110] Figure 26 The system described in 1900 helps shaders understand and correct differences in spatial colors by analyzing and correcting differences in average colors across multiple angles. This is beneficial for tools such as image-based color recipe retrieval systems used by shaders.

[0111] Figure 26 The system receives a target image containing target image data of the target coating. Color model 1902 and local color model 1906 contribute, at least in part, to the average generated by color model 1902. , , 1904 and color groups 1908 generated from the local color model 1906 are used to predict the color difference between the target coating and the sample coating 1914.

[0112] As provided by color model 1902, local color model 1906 predicts spatial color features (e.g., color entropy features, etc.) based on image data that is not “average” data. To generate color groups 1908, the color space is divided into primary, secondary, and tertiary color regions, as shown at 1910. As a non-limiting example, each pixel in an image can be classified based on the color family to which it belongs. Another non-limiting approach involves modeling the distribution of colors and determining which group a color belongs to. Such methods provide an overall characterization classification of the color groups. In implementations, local color models generate color groups by, for example, forming groups or clusters of similar color feature data. In implementations, the k-means clustering method can form primary color groups. , , Clustering of values, quadratic , , Clustering and cubic values , , Clustering of values. In this example, "k" in the k-means clustering method will be 3, resulting in three clusters.

[0113] The optimization routine 1912 uses the outputs from color model 1902 and local color model 1906 to generate a predicted color difference 1914 (e.g., ΔE). The optimization routine 1912 may include, for example, a numerical algorithm to generate a formulation that optimizes the predicted color difference relative to a standard. In an implementation, the numerical algorithm tests different formulations and determines which formulation has the smallest predicted color difference relative to a standard. Non-limiting examples include a function that generates a color difference metric based on the paint formulation, a model of the color phenomenon, and a color metric associated with a standard. The formulation that minimizes the function is selected. In this way, empirical models involving formulations and various localized color features can be used for color matching.

[0114] Different optimization routines can be used for the function, such as nonlinear numerical methods, linear numerical methods, etc. In this optimization routine, the function can be minimized to identify the minimum predicted color difference relative to a standard. The function can generate, for example, predictions of the average ΔE relative to a standard for color model 1902 and local color model 1906.

[0115] In this implementation, a constraint relaxation process is used to optimize the predicted color difference 1914. The constraint relaxation process may include: optimizing an objective function for the color difference that conforms to constraints whose coefficients are allowed to be relaxed. A penalty function is used within the color difference optimization routine to provide constraint relaxation. The optimized color difference determines the automotive paint composition used to spray the substrate at 1918. A model prediction 1916 with an acceptable ΔE can be sprayed, allowing an actual ΔE 1920 to be obtained. The sprayed sample 1922 is then compared to a standard 1924 to determine if an acceptable match has been achieved (e.g., the ΔE between the sample and the standard is within a pre-specified threshold, etc.). If the match is unacceptable, the process can be iterated until an acceptable match is achieved, such as... Figure 27 As shown.

[0116] Figure 27 The process flow for determining when a shader has achieved an acceptable match is described at 2000. The shader performs an initial spray at 2002 to attempt to match the target coating. Color space data is obtained using a color measurement device (e.g., a spectrophotometer).

[0117] The color difference 2004 between the initial spray and the target coating is fed as input to the biased color model at 2006 and to the local color model with prediction bias at 2008. The prediction bias can be determined based on the difference between the predicted ΔE and the actual ΔE obtained after the selected formulation is used to spray the sample. This bias is used to correct the prediction after the first iteration.

[0118] Both color models can be optimized in many different ways at 2010. In one implementation, a relaxation constraint method can be used to perform the optimization. The optimization results in a new predicted color difference 2012. The shader uses the predicted color difference 2012 in the paint formulation software to generate a new formulation to be sprayed at 2014. If the color difference 2016 from the additional spray is acceptable, as determined at 2018, the coloring process is completed at 2020. If the color difference is unacceptable, as determined at 2018, additional spraying and analysis iterations are performed until an acceptable color difference is obtained, as determined at 2018.

[0119] Figure 28 A system extended to analyze target coatings for texture matching is depicted at 2100. Texture matching can include any features describing local color phenomena. It should be noted that this does not include orange peel caused by sprayed paint.

[0120] In another implementation, texture matching is technically difficult because textures can vary over a given area, for example, not only with respect to grain size but also with respect to gloss, luminosity, turbidity, spots, speckles, glitter, or shimmer. Depending on the size and texture of the material's constituent parts, the texture to be matched can be described as the visual surface structure in the coating film plane. Texture does not include coating film roughness such as orange peel wrinkles, but includes visual irregularities in the coating film plane. Structures smaller than those visible to the human eye contribute to color, while larger structures contribute to texture.

[0121] The system examines the color group, texture features, and Δ“E” (color difference) across the target coating at 1920 to determine the grain size distribution across the target coating. Texture features are extracted at 1918 to match the texture characteristics of the coating film (sprayed at 1916) on the substrate to be matched.

[0122] It should be understood that texture machine learning models can compute paint formulations with matching texture characteristics. Machine learning models can, for example, examine the particle size distribution of the paint to be matched by checking dark and light areas in image data. Machine learning models such as convolutional neural networks can be applied to texture datasets to predict formulations based on ΔE measurements without deviating from acceptable color matching.

[0123] In this implementation, the convolutional neural network can be configured such that its convolution operators extract texture features from the input image. The configuration is established through supervised training, ensuring that it preserves the spatial relationships between pixels in the image. More specifically, the convolutional neural network learns optimal filter values, where the number of filters is set to extract a desired number of texture features. For example, if texture matching processing requires more texture features, the number of filters will be increased accordingly.

[0124] Figure 29 A server environment 2202 is depicted, in which a user 2202 can interact with a color and appearance matching analysis system 2204. The user 2202 can interact with the system 2202 in various ways, such as through one or more networks 2206. A server 2208 accessible via network 2206 can host the system 2204. One or more data storage devices 2210 can store color analysis data 2212 to be analyzed by the system 2204, as well as any intermediate or final data generated by the system.

[0125] Figure 29 The system can be a web-based integrated reporting and analysis tool that provides users with the flexibility and functionality to perform color and appearance matching analyses. It should be understood that the system can also be set up on a standalone computer for user access.

[0126] While at least one embodiment has been presented in the foregoing detailed description, it should be recognized that numerous variations exist. It should also be understood that one or more embodiments are merely examples and are by no means intended to limit the scope, applicability, or configuration. Rather, the foregoing detailed description will provide a convenient guide for those skilled in the art to implement the embodiments, and it should be understood that various changes can be made to the function and arrangement of the elements described in the embodiments without departing from the scope set forth in the appended claims and their legal equivalents.

Claims

1. A system for matching a target coating, comprising: Storage device, which is used to store instructions; One or more data processors are configured to execute instructions to: Receive a target image of the target coating, wherein the target image includes target image data; The target image data is subjected to feature extraction and analysis processing to determine the target image features, wherein the feature extraction and analysis processing includes dividing the target image into sub-images containing multiple target pixels; Determine the target pixel features of the sub-image, wherein determining the target pixel features of the sub-image includes determining the distribution of the target pixel features across the sub-image; A machine learning model is applied to generate predictions for identifying one or more types of patches present in the target coating, using the determined distribution of the target pixel features across the sub-images; and Color expert rules are applied to adjust the predictions associated with the identification of the one or more types of patches.

2. The system according to claim 1, wherein, Determining the target pixel features of the sub-image includes generating cumulative distribution statistics based on predetermined color attributes.

3. The system according to claim 1, wherein, The one or more data processors are configured to apply the feature extraction analysis process, wherein the feature extraction analysis process determines the individual target pixels of the target image. One or more of the color coordinates and the overall image color entropy of the target image.

4. The system according to claim 1, wherein, The one or more data processors are configured to execute instructions to retrieve a mathematical model for determining a matching sample image for computation, wherein the mathematical model is a machine learning model.

5. The system according to claim 1, wherein, Applying the color expert rules includes applying color analysis rules to the generated predictions to determine the probability that one or more of the output candidate images exist in the sub-image of the target coating.

6. The system according to claim 5, wherein, The generated prediction is adjusted based on an assessment of whether two patches typically do not exist for the target coating, according to the color analysis rules.

7. The system according to claim 1, wherein, The one or more data processors are configured to determine paint formulations corresponding to the calculated matching sample images, wherein the one or more data processors are configured to determine multiple paint formulations corresponding to the calculated matching sample images, wherein the multiple paint formulations include different grades of paint.

8. The system according to claim 1, wherein, The one or more data processors are configured to receive target image data of the target coating, wherein the target image data is associated with multiple images of the target coating having different light angles relative to the imaging device.

9. The system according to claim 1, wherein, The one or more data processors are configured to receive the target image data of the target coating, wherein the target image data is associated with multiple images of the target coating with different magnifications.

10. The system according to claim 1, wherein, The target coating is a metallic coating, a pearlescent coating, or a combination thereof.

11. A method for matching a target coating, comprising: A target image of the target coating is received by one or more data processors, wherein the target image includes target image data; The target image data is subjected to feature extraction and analysis processing by one or more data processors to determine target image features, wherein the feature extraction and analysis processing includes dividing the target image into sub-images containing multiple target pixels; The target pixel features of the sub-image are determined by the one or more data processors, wherein determining the target pixel features of the sub-image includes determining the distribution of the target pixel features across the sub-image; The machine learning model is applied through the one or more data processors to generate predictions for identifying one or more types of patches present in the target coating, using the determined distribution of the target pixel features across the sub-images; and Color expert rules are applied through one or more data processors to adjust predictions associated with the identification of the one or more types of slices.

12. The method according to claim 11, wherein, Determining the target pixel features of the sub-image includes generating cumulative distribution statistics based on predetermined color attributes.

13. The method according to claim 11, wherein, The one or more data processors are configured to apply the feature extraction analysis process, wherein the feature extraction analysis process determines the individual target pixels of the target image. One or more of the color coordinates and the overall image color entropy of the target image.

14. The method according to claim 11, wherein, The one or more data processors are configured to execute instructions to retrieve a mathematical model for determining a matching sample image for computation, wherein the mathematical model is a machine learning model.

15. The method according to claim 11, wherein, Applying the color expert rules includes applying color analysis rules to the generated predictions to determine the probability that one or more of the output candidate images exist in the sub-image of the target coating.

16. The method according to claim 15, wherein, The generated prediction is adjusted based on an assessment of whether two patches typically do not exist for the target coating, according to the color analysis rules.

17. The method according to claim 11, wherein, The one or more data processors are configured to determine paint formulations corresponding to the calculated matching sample images, wherein the one or more data processors are configured to determine multiple paint formulations corresponding to the calculated matching sample images, wherein the multiple paint formulations include different grades of paint.

18. The method according to claim 11, wherein, The one or more data processors are configured to receive target image data of the target coating, wherein the target image data is associated with multiple images of the target coating having different light angles relative to the imaging device.

19. The method according to claim 11, wherein, The one or more data processors are configured to receive the target image data of the target coating, wherein the target image data is associated with multiple images of the target coating with different magnifications.

20. The method according to claim 11, wherein, The target coating is a metallic coating, a pearlescent coating, or a combination thereof.

Images