A method and apparatus for generating robot path data for automatically coating at least a part of the surface of a spatial substrate with at least one coating material

Abstract

Aspects described herein generally relate to a method for generating robot path data for a robot path (or paths) that a robot including a coating tool follows while coating at least a portion of a surface of a spatial substrate with at least one coating material spatial substrate, and to respective apparatuses, or computer elements. Further, aspects described herein generally relate to a robot system for coating at least one surface of a spatial substrate with at least one coating material. The inventive method, respective apparatuses, or computer elements automate the application of coating material to substrates with large shape variations, providing consistency in application, as opposed to manual application, which is highly dependent on the painter applying the coating, for example, during the repair process of an automobile or automobile parts.

Description

Technical Field

[0001] Aspects described herein generally relate to a method for generating robot path data for a robot path that a robot including a coating tool should follow while coating at least a part of the surface of a spatial substrate with at least one coating material, and to respective apparatuses, or computer elements. Further, aspects described herein generally relate to a robot system for coating at least one surface of a spatial substrate with at least one coating material. More specifically, aspects described herein relate to a method for generating robot path data using different rule sets including color data and shape data of a spatial substrate, and rules for coating outer edges and edges adjacent to voids and rules for coating main surfaces, and to respective apparatuses, or computer elements. The use of 3D data that can be obtained by a robot using a scanning device attached to the robot makes it possible to determine the shape and color of the spatial substrate to be coated immediately before the coating process, and thus makes it possible to generate robot path data for substrates that exhibit a high level of variation in their shape without the need for such data to exist before performing the method of the present invention. Further, aspects described herein relate to a robot system including a computing device that generates robot path data according to the method of the present invention, and a robot device that receives the robot path data and coats a spatial substrate according to the received robot path data. The method of the present invention, each apparatus, or computer element can automate the application of a coating material to a substrate with a large variation in shape, and can provide a consistency of application that is highly dependent on a painter who applies the coating, in contrast to the manual application of a coating material during a repair process of an automobile or automobile parts, for example.

Background Art

[0002] Vehicles, especially land vehicles such as the bodies of automobiles, motorcycles, and trucks, are usually treated with multiple layers of paint to improve the appearance of the vehicle and protect it from corrosion, scratches, chips, ultraviolet rays, acid rain, and other environmental conditions. Multi-coat paint systems, including base coat layers and clear coat layers for automobiles and trucks, have been commonly used over the past 20 years.

[0003] The manufacture of such multi-coat paint systems generally involves electrophoretically depositing an electrocoat material on a metal substrate such as an automobile body and curing the applied electrocoat material. The metal substrate can be subjected to various pretreatments, for example, by applying a known conversion coating such as a phosphate coating, especially a zinc phosphate coating, before depositing the electrocoat. Thereafter, a filler or primer-surfacer material can be applied to and cured on the cured electrocoat. If such a layer is present, at least one base coat material containing color and / or effect pigments is applied over the cured layer. However, it is also possible to directly apply at least one base coat material (undercoat material) to the cured electrocoat layer. In the case of a plastic substrate, a primer can be applied before applying the base coat material to enhance the adhesion of the multi-layer coating to the substrate. The at least one base coat film or the topmost base coat film thus produced is then coated with a clear coat material without being cured separately. The clear coat film and all the base coat films are then cured together (so-called 2-coat 1-bake (2C1B) or 3-coat 1-bake (3C1B) process).

[0004] In such a multi - coat painting system, when film defects such as peeling, discoloration, and scratches occur, usually, these are repaired to restore the original appearance of the automobile. If such film defects occur directly after the OEM finish, these are repaired directly at the OEM manufacturing site within what is called "OEM automobile refinishing". If such defects occur at a later point in time, these are usually repaired at an automobile repair shop for what is called "automobile refinishing". The refinishing process is roughly classified into edge - to - edge repair, blend - in process, and spot repair. Edge - to - edge repair may be performed when the portion of the multi - layer coating to be repaired is relatively large, and usually, the damaged portion of the multi - layer coating is removed and the entire portion is refinished. Spot repair is performed when the portion of the multi - layer coating to be repaired is small or when the position of the portion of the multi - layer coating to be repaired is not conspicuous.

[0005] Overall refinishing generally includes washing and sanding, and, if necessary, filling the damaged part. After that, after further pretreatment, if necessary, the damaged part and the adjacent part are usually coated with an opaque coating agent such as an appropriate base - coat material. After drying the coating layer thus produced, the area adjacent to the coating layer is usually coated with a clear - coat composition and then dried together with the previously applied coating layer(s). Generally, spot repair includes sanding the spot to be repaired, coating the surface with an opaque coating material, drying the applied coating material, sanding the applied coating material, and applying a clear - coat material. To reduce the color mismatch of the repaired part and the spot, the repaired part is "blended" beyond the range of the repaired part and the spot. This is a process of reducing the build - up of the coating film of the applied coating layer as it moves away from the repaired part and the spot. Thus, it gradually changes from the (wrong) color of that part and the spot to the (correct) color of the remaining part. If this change is gradual enough, the human visual sense will not feel a mismatch.

[0006] When not only the multi-layer coating but also the substrate (lower substrate) is damaged, for example, in a car accident, it is necessary to remove the damaged parts and attach new parts with the required colors during the repair of the car. Therefore, before attaching parts to a car, it is necessary to apply a multi-layer coating to the entire part.

[0007] Today, the requirements imposed on the refinishing of cars are extremely high. Therefore, both visually and technically, the finish must be equivalent to the original baked finish. That is, the color of the repaired part must match the color of the other parts of the vehicle so that the observer cannot identify the repaired part. Furthermore, the mechanical properties of the repaired multi-layer coating must be equivalent to those of the original finish.

Summary of the Invention

Problems to be Solved by the Invention

[0008] However, the optical appearance of the applied coating material depends greatly on the parameters used during the application of the coating material. That is, changes in the application conditions result in changes in the optical overview of the car or car part. In order to reduce the influence of the application conditions on the optical appearance of the coated substrate, a robot system that automatically applies the coating material to the substrate can be used. In a robot painting system, a control device gives instructions (also called robot path data) to the robot, and the robot paints each substrate according to the provided robot path data.

[0009] When generating complex-shaped robot movements for spatial substrates such as cars and their parts, the painting must be as uniform and complete as possible. The generation of the above-mentioned complex robot movements can be automatically performed based on electronic data such as CAD data of the substrate, such as a car bonnet, a car fender, a car door, etc., and painting rules defined in advance for each type of substrate. However, the above method cannot be used for substrates for which CAD data is not available, for example, substrates used in the field of car repair that have a high level of variation in their shape.

[0010] Accordingly, it is desirable to provide a method and system that enables the automatic generation of robot path data, i.e., without human intervention, for a spatial substrate having a high level of variation in its shape, in order to enable the application of coating material and the consistency of the resulting optical coating quality.

Means for Solving the Problem

[0011] Overview In order to solve the above problems, the following is proposed: A computer-implemented method for generating robot path data for a robot path followed by a robot including a coating tool when coating at least a part of the surface of at least one spatial substrate with at least one coating material, the method including the following steps: (a) Providing, via a communication interface, to at least one computer processor, - Spatial substrate data for each spatial substrate, including substrate classification data and data indicating the shape and color of each spatial substrate, and - Coating material data including data indicating the type of at least one coating material, and optionally data indicating the order of the coating material applied to each spatial substrate, and / or data indicating the coating tool, the step of providing; (b) Optionally, determining, by at least one computer processor, whether each spatial substrate includes at least one masking material based on the provided spatial substrate data; (c) Using at least one computer processor to search via a communication interface, - Coating tool parameter data based on the provided coating material data, the coating tool parameter data including coating tool tolerance data and at least one application parameter associated with at least one coating material, the coating tool parameter data, - Coating procedure data including a rule set for coating outer edges and edges adjacent to open spaces, and a rule set for coating main surfaces - Creating substrate type data based on provided spatial substrate data, the substrate type data including a rule set regarding the type of spatial substrate that matches the substrate classification data; (d) Generating tool path data of the tool path traced by a coating tool along the surface of each spatial substrate by at least one computer processor based on the data obtained in step (c) and optionally the result of the determination executed in step (b); (e) Generating robot path data by at least one computer processor based on the tool path data generated in step (d); (f) Providing the generated robot path data via a communication interface.

[0012] The following configuration is further disclosed: A method of coating at least a part of the surface of a spatial substrate with a coating material using a robot system including a robot including a coating tool, the method including the following steps: (a) Providing, via a communication interface, to at least one computer processor - Spatial substrate data of the spatial substrate including substrate classification data and data indicating the shape and color of the spatial substrate - Coating material data including data indicating the type of at least one coating material, and optionally data indicating the order of the coating materials applied to the spatial substrate, and / or data indicating the coating tool (b) Optionally, determining by at least one computer processor whether the spatial substrate includes at least one masking material based on the provided spatial substrate data; (c) Using at least one computer processor to, via a communication interface, the following - Coating tool parameter data based on the provided coating material data, wherein the coating tool parameter data includes coating tool tolerance data and at least one application parameter related to the coating tool, the coating tool parameter data, - Coating procedure data including a rule set for coating outer edges and edges adjacent to at least one open space and a rule set for coating the main surface, - Substrate type data based on the provided spatial substrate data, wherein the substrate type data includes a rule set regarding the type of the spatial substrate that matches the substrate classification data, the substrate type data, The step of searching, (d) Using at least one computer processor, based on the data obtained in step (c) and optionally the result of the determination executed in step (b), to generate tool path data of at least one tool path traced by a coating tool along the surface of the spatial substrate, wherein generating the tool path data of at least one tool path traced by a coating tool along the surface of each spatial substrate includes the following: - Generating a 3D model of the spatial substrate by at least one computer processor based on the provided spatial substrate data and, optionally, applying a rule set for smoothing the surface of the generated 3D model, - Determining, by the computer processor, one or more outer edges, one or more edges adjacent to open spaces, the main surface, and one or more open spaces existing within the spatial substrate based on the obtained coating procedure parameter data and the generated and optionally smoothed 3D model, - Generating tool path data for the outer edge and tool path data for at least one edge adjacent to an open space based on the retrieved coating procedure data, the retrieved coating tool parameter data, the retrieved substrate type data, the determined outer edge, the edge and the open space adjacent to the open space, and the generated and optionally smoothed 3D model by at least one computer processor; - Generating tool path data for the main surface of each spatial substrate based on the retrieved coating procedure data, the retrieved coating tool parameter data, the determined main surface, and the generated and optionally smoothed 3D model by at least one computer processor; - Optionally repeating the above steps for at least one additional coating material based on the retrieved coating procedure data and the retrieved coating tool parameter data; Steps including: (e) Generating robot path data with at least one computer processor based on the tool path data generated in step (d); (f) Providing the generated robot path data to a robot including a coating tool for coating at least a portion of the surface of the spatial substrate with a coating material via a communication interface.

[0013] The essential advantage of the method according to the invention is that by using a set of rules for coating the outer edges and the edges adjacent to the open space, a set of rules for coating the main surfaces, and a set of rules for each type of spatial substrate, it is possible to generate robot path data for spatial substrates having a high level of variation in their shape. By means of the set of rules, different coating rules can be applied to the outer edges of the spatial substrate, the edges adjacent to the open space, the identified features such as the open space, and / or the features of the spatial substrate. Thereby, using a robot painting system, it is possible to paint spatial substrates such as automotive parts having a high level of geometric variation in the field of automotive refinishing. Depending on the information regarding the color of the spatial substrate, specific regions of the spatial substrate that are excluded from the coating process (e.g., masking material) or included (e.g., primer or regions already coated with a primer-surface coating) can be automatically identified. Furthermore, based on the color information, the boundaries of these colored regions can be identified, thereby enabling a smooth coating transition to improve the color and surface quality of the coated spatial substrate by applying specific rules. The method according to the invention is capable of generating robot path data for one or more spatial substrates in one step, regardless of the coating material applied to the spatial substrate and the type of spatial substrate. Thus, the method according to the invention is very effective with respect to fully automatically coating a plurality of spatial substrates with a coating material. In manual application, variations occur due to the influence of various application parameters on the overall quality obtained, but by automating the application of the coating material, a consistently high optical and mechanical quality is achieved.

[0014] Furthermore, the following configuration is disclosed: A computing device for generating robot path data for a robot path followed by a robot including a coating tool when coating a spatial substrate with at least one coating material, the following: - At least one computer processor, and - A memory that stores instructions that, when executed by a processor, configure the apparatus to perform the steps of the method of the present invention, A computing device including the same.

[0015] The following further configuration is disclosed: A robot system for coating at least one surface of a space substrate with at least one coating material, comprising: - A computing device of the present invention for generating robot path data for a robot path that a robot of the robot system follows when coating at least one surface of at least one space substrate with at least one coating material, - A robot device configured to receive the generated robot path data and apply at least one coating material from a coating tool to at least a part of the surface of the at least one space substrate using the received robot path data, A robot system including the same.

[0016] The following further configuration is disclosed: A robot system for coating at least one surface of a space substrate with at least one coating material, comprising: - A computing device for generating robot path data for a robot path that the robot of the robot system follows when coating at least one surface of the space substrate with at least one coating material, the computing device comprising: · At least one computer processor, and · A memory that stores instructions that, when executed by the processor, configure the apparatus to perform the method disclosed herein, - A robot device configured to receive the generated robot path data and apply at least one coating material from a coating tool to at least a part of the surface of the space substrate using the received robot path data, A robot system including

[0017] The essential advantage of the robot system according to the invention is that a spatial substrate having a high level of variation with respect to its geometric shape can be coated with at least one coating material completely automatically without the need for user operation during the generation of the robot path data. By using a rule set during the generation of the robot path data, such robot path data can be reliably generated for substrates having a high level of geometric variation, and thus, regardless of the geometric shape of each substrate, the generated robot path data can be used by the robot system to coat such substrates so as to obtain coatings having high optical and mechanical quality. If the robot system further comprises a scanning device, data regarding the color and shape of the spatial substrate can be determined by the robot device, thus reducing the amount of separate devices required to determine the shape and color of the spatial substrate required by the computing device of the invention to determine the robot path data. Further, due to the presence of the scanning device, data regarding the color and shape of the spatial substrate to be coated can be acquired before the coating process, so that spatial substrates for which data regarding their color and shape is not available prior to performing the coating process can be coated. Information regarding the color of the spatial substrate can be used to automatically identify specific regions of the spatial substrate that are excluded from the coating process (e.g., masking material) or included (e.g., primer or regions already coated with a primer-surfacer coating). Further, the color information can be used to identify the boundaries of those colored regions and apply specific rules to enable a smooth transition of the coating for improving the color and surface quality of the coated spatial substrate. The robot system according to the invention can apply at least one coating material to one or more spatial substrates regardless of the coating material applied to the spatial substrate and the type of the spatial substrate, and thus, the robot system is very effective with respect to the fully automatic application of the coating material to a plurality of spatial substrates.

[0018] Furthermore, the following configurations are disclosed: Use of a method or a computing device according to the invention for coating at least a part of the surface of at least one spatial substrate with at least one coating material using a robot comprising a coating tool.

[0019] Furthermore, the following configurations are disclosed: A non-transitory computer-readable storage medium, which when executed by a computer, includes instructions for causing the computer to execute steps according to the method of the invention.

[0020] The disclosures and embodiments described herein relate to the methods, systems, devices, and computer elements disclosed herein, and vice versa. The advantages provided by any of the embodiments and examples provided herein apply equally to all other embodiments and examples, and vice versa.

Mode for Carrying Out the Invention

[0021] The method of the present invention enables the generation of robot path data for a robot path that a robot including a coating tool such as a spray coating tool follows while coating at least a part of the surface of at least one spatial substrate with at least one coating material. The generated robot path data may be used by a robot system including a coating tool for coating at least a part of the surface of the substrate. In one embodiment, the robot path data is generated for one spatial substrate. In another example, the robot path data is generated for at least two spatial substrates, for example 2 to 5 or 2 to 10 spatial substrates. As used herein, the term "robot path" refers to the relative movement path of a robot such as an industrial robot with respect to the surface of a spatial substrate, particularly the surface of the spatial substrate to be coated with a coating material applied from a coating tool attached to the robot. As used herein, the term "coating material" refers to a chemical composition in the form of a liquid, paste, or powder that, when applied to the surface of a spatial substrate, produces a coating having protective, decorative, and / or other specific properties (see DIN EN 971-1:1996-09). The coating to be produced may comprise one coating layer or multiple coating layers (also referred to as a multi-coat paint system). The coating layers of the coating can be produced using different or similar coating materials. For example, the coating can comprise two base coat layers prepared by applying two different base coat materials or by applying the same base coat material twice.

[0022] The robot can be any multi-axis industrial robot suitable for applying at least one coating material to the surface of a spatial substrate. The robot can include a movable robot member such as a robot arm, and the coating tool may be attached to the movable robot member. The robot can be controlled using a robot control device described later.

[0023] The spatial substrate can be of any shape. In one aspect, the spatial substrate is a vehicle part. The term "vehicle part" should be understood in a broad sense in this embodiment and relates to parts of automobiles, vans, minivans, buses, SUVs (sports utility vehicles); trucks; semi-trucks; tractors; motorcycles; trailers; ATVs (all-terrain vehicles); pickup trucks; large moving bodies such as bulldozers, mobile cranes, earthmovers, etc.; airplanes; boats; ships; and other means of transportation. Particularly preferably, the term "automobile part" represents a part of an automobile; a truck; a semi-truck; a tractor; a motorcycle; a trailer; an ATV (all-terrain vehicle); a pickup truck or a large moving body. Preferably, the automobile part is a body part, particularly a bonnet, fender, door, bumper, quarter panel, trunk or hatch.

[0024] The spatial substrate may be an uncoated spatial substrate, i.e., a spatial substrate that does not contain any coating layer, or a spatially substrate that is at least partially coated, i.e., a spatial substrate in which at least a part of the surface of the substrate already contains at least one coating layer such as a dried or cured coating layer. Suitable spatial substrates include: (i) an uncoated or at least partially coated spatial metal substrate; (ii) an uncoated or at least partially coated spatial plastic substrate; and (iii) an uncoated or at least partially coated spatial substrate containing a metal part and a plastic part. Suitable metal substrates include the group consisting of steel, iron, aluminum, copper, zinc, and magnesium substrates and substrates composed of alloys of steel, iron, aluminum, copper, zinc, and magnesium, or are selected from the group consisting of these. The metal substrate can be pretreated by methods known per se, i.e., for example, it can be washed and / or provided with a known conversion coating. Washing can be carried out mechanically, for example, by means of wiping, grinding, and / or polishing, and / or chemically, for example, by means of an etching method such as surface etching in an acid or alkali bath using hydrochloric acid or sulfuric acid, or by means of washing with an organic solvent or an aqueous detergent. The pretreatment can be carried out by applying a conversion coating, in particular a phosphate treatment and / or a chromate treatment, preferably a phosphate treatment. Preferably, the spatial metal substrate is treated by at least a conversion coating, in particular a phosphate treatment, preferably a zinc phosphate treatment. Preferred spatial plastic substrates are (i) polar plastics such as polycarbonate, polyamide, polystyrene, styrene copolymers, polyester, polyphenylene oxide, and blends thereof, (ii) synthetic resins such as polyurethane RIM, SMC, BMC, etc., and (iii) polyethylene and polypropylene type polyolefins with a high rubber content such as PP-EPDM, and surface-activated polyolefin substrates, or are substrates comprising or consisting of these. The spatial plastic substrate may further be fiber-reinforced, in particular using carbon fibers and / or metal fibers.

[0025] The coating material(s) used to coat at least a part of the surface of the spatial substrate can be any suitable liquid or solid coating material(s), such as primer-surfacer coating material(s), primer coating material(s), basecoat material(s), clearcoat material(s), topcoat material(s), or single-stage material(s), etc. The "primer-surfacer coating material" (also called filler coating material) represents a coating material used to prepare an intermediate layer designed to fill the unevenness on the surface of the spatial substrate, which supports corrosion resistance and adhesiveness and protects against mechanical exposure such as chipping. The "primer coating material" refers to a coating material used to prepare the first coating layer of a multi-layer coating on the surface of the spatial substrate. The primer coating material is used to improve the adhesiveness of the multi-layer coating. Further, the primer coating material can result in a coating layer that can provide improved corrosion protection, for example, on a spatial substrate made of metal. The "basecoat material" represents a color-imparting intermediate paint commonly used in automotive painting. The basecoat material can be formulated as an effect coat material or a solid color coat material. The effect coating material generally contains at least one effect pigment and optionally other coloring pigments or spheres that give the desired color and effect, and the solid coating material contains only coloring pigments and no effect pigments. The "clearcoat material" represents a transparent coating material. "Transparent" means that the film formed from the coating material is not completely opaque, but instead has a certain degree of transparency such that the color of the underlying coating layer can be seen through the clearcoat layer formed from the clearcoat material. Therefore, the clearcoat material may contain no pigments at all, only transparent pigments, or an amount of pigments that does not make the coating layer obtained from the clearcoat material opaque. Typically, the clearcoat material is applied over the basecoat layer formed from the basecoat material to protect the underlying basecoat layer.The term "topcoat material" refers to a coating material that is applied as the final coating material in a coating process, and the coating layer formed from that material becomes the topmost coating layer of the coating. The topcoat material can be a colored coating material, i.e., a coating material containing an effect pigment and / or a coloring pigment, or it can be a clearcoat material. The term "single-layer material (single-stage material)" refers to a colored or transparent coating material that is applied in a single layer on a spatial substrate, i.e., these do not require the application of a further coating material on top of the above layers. The robot path data can be generated for one coating material or for multiple coating materials, depending on whether the resulting coating should be a single-layer coating or a multi-layer coating, and also depending on the coating layers already present on the spatial substrate.

[0026] In one aspect, the robot is disposed within a spray booth, and each spatial substrate is disposed within the working space of the robot. The term "working space of the robot" should be understood broadly in this instance and relates to the set of all positions that the robot, particularly the movable robot members, can reach. The working space of the robot generally depends on many factors, including the dimensions of the movable robot members such as the robot arm. Particularly preferably, the robot is attached to a rail system such as a Güdel rail system installed on the side and / or ceiling of the spray booth to enable movement of the robot along the spatial substrate(s) in order to expand the working space of the robot and enable the coating of spatial substrate(s) having larger dimensions. In one embodiment, the working space of the robot corresponds to the dimensions of the spray booth. In another example, the working space of the robot is smaller than the dimensions of the spray booth. The spray booth may include markings indicating different zones. Thereby, the user may be able to place the spatial substrates within each zone or within each region of the zones within the spray booth.

[0027] In one aspect, the coating tool includes a coating material applicator, particularly a spray applicator. In this example, the coating tool further includes a coating material reservoir configured to contain a specific coating material, and the reservoir is attached to the coating material applicator. In this embodiment, the coating material reservoir is directly attached to the material applicator. Direct attachment means that the reservoir is directly connected to the material applicator, for example, by using a suitable connection. This can avoid the use of additional tubes present in the spray booth that need to be considered during the generation of robot path data. In another embodiment, the reservoir is attached to the material applicator via a tube. In this example, the reservoir is located inside or outside the spray booth and is connected to the material applicator via a tube. The coating tool may be permanently attached to the robot or temporarily attached. When the coating tool is temporarily attached to the robot, multiple different coating tools, for example, coating tools including coating material reservoirs containing different coating materials, can be stored in a tool rack as described later. Before painting, the robot may be instructed by a robot controller to select an appropriate coating tool from the tool rack as described later.

[0028] Step (a): In step (a) of the method of the present invention, spatial substrate data and coating material data are provided to at least one computer processor via a communication interface. The spatial substrate data includes substrate classification data and data indicating the shape and color of each spatial substrate, in particular data indicating the surface of each spatial substrate. When one or more spatial substrates are coated with at least one coating material, the spatial substrate data of each spatial substrate to be coated is provided in step (a). The coating material data includes data indicating the type of at least one coating material. In one example, the coating material data further includes the order of the coating materials applied to the spatial substrate. This is preferable to ensure that the coating materials are applied in the correct order when multiple coating materials are applied to the spatial substrate. In another example, the coating material data does not include the order of the coating materials applied to the spatial substrate. This is preferable when there is only one coating material to be applied to the spatial substrate or when a predefined order of coating materials is used during the generation of tool path data.

[0029] The term "communication interface" should be understood broadly in this case and relates to software and / or hardware interfaces for establishing communication such as the transfer and exchange of signals and data. The software interface may be, for example, a function call or an API. The communication interface may include a transceiver and / or a receiver. The communication may be wired or wireless. The communication interface may be based on or support one or more communication protocols. The communication protocols may be, for example, short-range communication protocols such as Bluetooth(R), WiFi, or long-range communication protocols such as cellular or mobile network wireless protocols such as the second-generation mobile phone network ("2G"), 3G, 4G, LTE (Long-Term Evolution), or 5G. Alternatively, in addition to that, the communication interface may be based on its own short-range or long-range protocol. The communication interface may support any one or more standard protocols and / or proprietary protocols.

[0030] The term "computer processor" (hereinafter also referred to as "hardware processor") should be understood in a broad sense in this embodiment, and relates to any logic circuit configured to execute the basic operations of a computer or system, and / or generally, a device configured to execute computational or logical operations. In particular, the computer processor may be configured to process the basic instructions that drive a computer or system. As an example, the computer processor may include at least one arithmetic logic unit ("ALU"), at least one floating-point arithmetic unit ("FPU") such as a math coprocessor or a numeric coprocessor, a plurality of registers, specifically, registers configured to supply operands to the ALU and store the operation results, and memories such as L1 cache memory and L2 cache memory. In particular, the computer processor may be a multi-core processor. Specifically, the computer processor may be a central processing unit ("CPU") or may include a central processing unit. The computer processor may also be a graphics processing unit ("GPU"), a tensor processing unit ("TPU"), a Complex Instruction Set Computing ("CISC") microprocessor, a Reduced Instruction Set Computing ("RISC") microprocessor, a Very Long Instruction Word ("VLIW") microprocessor, or a processor implementing other instruction sets, or a processor implementing a combination of instruction sets. The computer processor may also be one or more special-purpose scanning devices such as an application-specific integrated circuit ("ASIC"), a field-programmable gate array ("FPGA"), a complex programmable logic device ("CPLD"), a digital signal processor ("DSP"), a network processor, etc. The method of the present invention may be implemented as software in a DSP, a microcontroller, or other side processors, or as a hardware circuit in an ASIC, a CPLD, or an FPGA.The term "computer processor" may represent one or more scanning devices, such as a distributed system of scanning devices arranged across multiple computer systems (e.g., cloud computing), and is not limited to a single device (apparatus) unless otherwise specified.

[0031] Spatial substrate data: In one aspect, the data indicating the shape and color of each spatial substrate includes data representing each spatial substrate in three-dimensional space, particularly the three-dimensional point cloud of each spatial substrate, and the color data of each spatial substrate. The color data may include color space data or still image data. An example of color space data is defined by L*a*b*, where L* represents lightness, a* represents the red / green appearance, and b* represents the yellow / blue appearance. Another example of color space data is defined by L*, C*, h, where L* represents lightness, C* represents chroma, and h represents hue. Yet another example of color space data is defined by RGB, where R represents the red channel, G represents the green channel, and B represents the blue channel. Particularly preferably, the color space data is defined by RGB. The still image data may represent a frame acquired by a scanning device described later.

[0032] Separate from the spatial substrate classification data and the data indicating the shape and color of each spatial substrate, the spatial substrate data may further include a substrate ID, a substrate name, a barcode, a QR code (registered trademark), data related to the substrate manufacturer, data related to the substrate composition, substrate manufacturing data, a rule set for smoothing the surface of a 3D model generated from the data indicating the shape and color of each spatial substrate, or a combination thereof. By associating a rule set for smoothing the surface of a 3D model with a defined type of spatial substrate, it is possible to apply different rule sets to different substrate types, define specific smoothing rules for each substrate type, and perform smoothing according to the substrate type.

[0033] The spatial substrate data may be provided in many ways. In a non-limiting example, the provision of spatial substrate data includes the following: - Detecting, by at least one computer processor, a user input indicating a substrate classification associated with each spatial substrate and a user input indicating the position of each spatial substrate within the working space of the robot; - Determining, by at least one computer processor, based on the detected user input, the substrate classification data of each spatial substrate and the position of each spatial substrate within the working space of the robot; - Providing, via a communication interface, data of the working space of the robot to at least one computer processor; - Determining, by at least one computer processor, based on the provided data of the working space of the robot, the collision geometry existing within the working space; - Determining, by at least one computer processor, based on the determined position of each spatial substrate within the workspace of the robot and the determined collision geometry, scan path data for a scan path to be traced by a scanning device along the surface of each spatial substrate, and providing the determined scan path data to the scanning device via a communication interface; - Obtaining, by at least one computer processor, via a communication interface, data indicating the shape and color of each spatial substrate acquired by a scanning device based on the provided scan path data, and generating spatial substrate data for each spatial substrate by combining the acquired data with at least the determined substrate classification data of each spatial substrate.

[0034] The step of detecting user input and the step of determining the substrate classification data and the positions of each spatial sub-state based on the detected user input can also be executed after any point in time before generating the spatial substrate data. For example, one or both of the steps may be executed after determining the scan path data and before generating the spatial substrate data. Detecting user input can include, for example, displaying a user interface that enables the user to select the substrate classification associated with each spatial substrate to be coated by displaying a list of possible substrate classifications such as the trunk, hood, fender, etc., and also, for example, by displaying a map or image of the spray booth and prompting the user to select the position of each spatial substrate to be coated within the displayed map of the spray booth, the position of each spatial substrate to be coated within the working space of the robot can be selected. The graphical user interface may be displayed on the screen of the display device. In one example, the display device houses at least one computer processor, for example when the display device is a tablet or the like. In another example, the display device is attached to at least one computer processor via a communication interface, for example when the display device is an external computer monitor, a laptop monitor, or when the display device simply serves to display a graphical user interface or the like. User input can be detected via an input device or an input / output device, particularly interaction elements such as a mouse, keyboard, trackball, touch screen, or a combination thereof. In another example, the interaction element may be a projection area where user input in the form of gestures such as finger gestures or hand movements is received.

[0035] In one example, the position of each spatial substrate in the working space of the robot includes the position of each spatial substrate in at least one zone of the spray booth including the robot. Therefore, the spray booth is separated into at least two different zones, and the user needs to select the zone(s) of the spray booth where each spatial substrate is located. The selection of at least one zone can be easily done, for example, by using a graphical user interface that displays a map or image of the spray booth separated into at least two different zones and prompts the user to select at least one zone, such as clicking on each zone or the number of zones, or entering the number assigned to each zone.

[0036] The data of the working space of the robot can be provided by obtaining the data from a data storage medium such as an internal memory or database connected to at least one computer processor via a communication interface, or by performing a scan of the spray booth including the spatial substrate(s) to be coated. In one embodiment, the data includes data representing the working space in a three-dimensional space, particularly the three-dimensional point cloud of the working space, and the color data of the working space. The three-dimensional point cloud of the working space can be obtained, for example, by attaching a generally known three-dimensional scanning tool to the robot and scanning the working space of the robot using the tool. The color data of the working space is necessary for performing step (b) of the process of the present invention and is also necessary for determining an existing coating layer on a substrate that may require the application of specific values included in the coating tool parameter data in order to obtain a smooth transition between the existing coating layer and the coating material applied by the coating tool of the robot.

[0037] The generation of collision geometry from the provided data of the robot's working space can be performed according to methods well known in the art, including filtering the provided data to reduce the number of data points, determining whether the data contains geometries of a specific size, generating 3D object(s) present in the working space from extreme data points, and filling the generated object(s) with volume, or combinations thereof.

[0038] Data regarding the working space of the robot is only sufficient to determine the collision geometry. Therefore, in order to enable the generation of tool path data described later, it is necessary to determine scan path data for obtaining detailed data regarding the geometry and color of each spatial substrate existing within the working space of the robot. In one embodiment, scan path data is determined for each zone of the spray booth from the positions of each spatial substrate within the working space of the robot and the determined collision geometry. When the spatial substrate exists in a plurality of zones of the spray booth, when determining the scan path data, the zones may be passed through in a defined order. The scan path data preferably consists of a series such as a raster of data points in the space, and each series represents a full-stroke line between the start point and the end point of a raster stroke along the surface of each spatial substrate to be scanned. The determined scan path data is supplied to a scanning device (hereinafter also referred to as a sensor device or a sensor system) via a communication interface. When the scanning device is attached to the robot, the determined scan path data is preferably provided to a robot controller connected to the robot via a communication interface, and the robot controller is configured to control the robot using the received scan path data. When the scanning device exists separately from the robot, the scan path data is provided to the scanning device or a controller configured to control the scanning device. Attaching the scanning device to the robot that executes the application of the coating material and providing the generated scan path data to the robot controller is preferable because it reduces the number of devices necessary for generating spatial substrate data and for automatically coating the spatial substrate, and thus reduces the complexity.

[0039] The generated spatial substrate data can be associated with a substrate ID and provided to and stored in a data storage medium via a communication interface. Thereby, for example, during the generation of tool path data or when the method of the present invention is executed on the same spatial substrate at a later time, the spatial substrate data generated using the substrate ID at a later time can be retrieved. As a result, when the data is subsequently required, generation of the spatial substrate data can be avoided. When there are a plurality of spatial substrates within the workspace of the robot, the spatial substrate data generated for each spatial substrate is associated with an appropriate substrate ID. This may be performed, for example, by assigning a position ID to the scan path data acquired for each spatial substrate and associating the position ID with the position of each spatial substrate determined based on the detected user input.

[0040] Coating material data: The coating material data includes at least data indicating the identity of the coating material, and may further include data indicating the order of the coating materials applied to each spatial substrate and / or data indicating the coating tool used to apply each coating material. The data indicating the identity of the coating material preferably includes the name of each coating material type, the ID of each coating material type, or a combination thereof. The name of the coating material type may include, for example, clear coat, base coat, sealer, primer, primer surfacer, top coat, one-component material, etc. Each coating material type may be assigned a unique ID so that it can be identified by the uniquely assigned ID. The data indicating the coating tool may include a unique coating tool ID, a coating tool type such as an ESTA, an air applicator, or a combination thereof.

[0041] Data on the coating material can be provided, for example, by displaying a user interface and prompting the user to input data indicating the type of coating material (such as clear coat, base coat, sealer, primer, primer surfacer, top coat, single-stage material, etc.). Identification of the coating material can also be provided by displaying a list of existing coating material types within the user interface and detecting a user input indicating selection of at least one list item. The user may input or provide the ID / barcode / QR code of each coating material, and then the processor may obtain the data indicating the identification from a data storage medium in which the ID / barcode / QR code of the coating material and the respective data indicating the identification of the type of coating material are stored in an associated state.

[0042] In one example, the order of the coating materials to be applied needs to be input by the user, for example, by displaying a user interface and prompting the user to input the order of the identification (identity) of the provided coating materials. In another example, a predefined order is used for the provided data indicating the identification of the coating materials. The predefined order may be determined from a rule set including rules for determining the order of the coating materials to be applied based on, for example, the types of the provided coating materials. For example, the rule set may include a rule that determines that the coating material types "base coat" and "clear coat" input by the user are applied in the order that the clear coat follows the base coat.

[0043] In one embodiment, the data indicating the coating tool must be input by the user, for example, by displaying a user interface to prompt the user to input data, for example, by displaying a list to prompt the user to select a list item, or by displaying a text field to prompt the user to input appropriate data. This may be preferable when the m coating materials are applied with multiple coating tools. The reason for this is that in this case, the data indicating the type of at least one coating material cannot be used to search for the coating parameters related to a specific coating tool because multiple coating tools can be used.

[0044] In certain aspects, the coating material data further includes chemical property data of the coating material, physical property data of the coating material, data regarding the composition of each coating material, the ID of each coating material, the barcode of each coating material, the QR code of each coating material, data regarding the material manufacturer of each coating material, data indicating the spatial substrate to be coated with the coating material, or a combination thereof. The data indicating the spatial substrate to be coated may include the aforementioned substrate ID or substrate type. This data is generally only necessary when multiple spatial substrates are coated and the coating materials used to coat the multiple spatial substrates are different for at least a portion of the spatial substrates. In this case, the data indicating the spatial substrate to be coated with the coating material needs to be provided to ensure that the correct coating material is applied to each spatial substrate.

[0045] Optional step (b): In step (b), the computer processor determines, based on the data provided in step (a), whether each spatial substrate, particularly whether a part of the surface of each spatial substrate contains at least one masking material. By using a masking material on a part of the surface of the spatial substrate, coating of the masked area can be avoided, which can be used, for example, when the spatial substrate should not be coated in certain areas, or when the spatial substrate is coated with two different colored coating materials and there is a distinct visible separation between the applied coating materials. When this step is executed, the area covered with the masking material is excluded when determining the robot path data, so that the application of the coating material to the masking material is avoided, and the consumption and overspray of the coating material can be reduced.

[0046] The term "masking material" should be understood broadly here and relates to a material applied to a part of the surface of a spatial substrate to protect the surface from the coating material applied onto the spatial substrate using a coating tool attached to a robot. Thus, the surface of the spatial substrate covered with the masking material is not coated with the applied coating material. The masking material should have sufficient adhesion on the surface of the spatial substrate to avoid removal of the masking material during the coating process. However, the masking material should be removable without leaving unnecessary residues on the surface of the spatial substrate to avoid time-consuming cleaning operations after removal.

[0047] In one aspect, the masking material includes masking paper, masking tape, masking film, or a combination thereof. For example, masking paper can be fixed to a part of the surface of the spatial substrate using masking tape or masking film.

[0048] Based on the provided spatial substrate data, determining whether each spatial substrate includes at least one masking material may include generating a three-dimensional (3D) model(s) using the provided spatial substrate data and providing the generated 3D model(s) to a parameterized data-driven model in the historical data of the masking material, particularly the historical 3D model of the spatial substrate including the masking material(s). Preferably, the 3D model is created using 3D point cloud data and color data included in the provided spatial substrate data. Generation of the 3D model from the point cloud data and color data can be performed according to methods well-known in the art.

[0049] A "data-driven model" represents a model derived at least in part from data. By using a data-driven model, relationships that cannot be modeled by physical and chemical laws can be described. By using a data-driven model, relationships can be described without solving equations from physical and chemical laws. This can reduce the amount of computation and improve the speed. The data-driven model may be derived from machine learning (Machine Learning and Deep Learning frameworks and libraries for large-scale data mining: a survey, Artificial Intelligence Review, Vol.52, 2019, pages 77 to 124). The data-driven model may include an empirical model or a so-called "black box model". The empirical model or "black box" model may represent a model constructed using one or more of machine learning, deep learning, neural networks, or other forms of artificial intelligence. The empirical model or "black box" model may be any model that provides a good fit between the training data and the test data.

[0050] In one example, a data-driven model is a machine learning model trained to determine a masking material present on the surface of a spatial substrate based on historical color data of the masking material present on the surface of past spatial substrates. The past color data may include a 3D color model of the spatial substrate, where each spatial substrate includes at least one masking material. The trained machine learning module may be a classifier that classifies which surface regions of the provided 3D color model include a masking material. "Machine learning" may represent a computer algorithm that improves through experience and uses supervised, unsupervised, or semi-supervised machine learning techniques to build a model based on sample data, often described as training data. Supervised learning involves using training data with known labels or outcomes, where prediction is required and the model is prepared through a learning process that corrects the prediction if it is incorrect. The training process continues until the model achieves a desired level of accuracy for the training data. Semi-supervised learning involves using a mixture of labeled and unlabeled input data and preparing the model through a learning process where the model must learn a structure to organize the data as well as make predictions. Unsupervised learning involves using unlabeled input data with no known outcomes and preparing the model by inferring structures such as common rules or similarities present in the input data. In one example, the machine learning algorithm is trained by selecting inputs and outputs to define the internal structure of the machine learning algorithm, applying a collection of input and output data samples to train the machine learning algorithm, applying input data samples that include known masking material(s) to verify the accuracy of the machine learning algorithm, comparing the generated output values to the expected output values, and modifying the parameters of the machine learning algorithm using an optimization algorithm if the received output values do not match the input data samples. As input data, the aforementioned images of past (historical) spatial substrates with or without a masking material may be used.The input data is randomly selected, provided that the training data contains the complete spectrum of the masking material. The output may be a 3D model of the spatial substrate indicating the area containing the masking material, or a classifier in the case where the spatial substrate does not contain the masking material.

[0051] In principle, a suitable machine learning model or algorithm can be selected by a person skilled in the art considering preprocessing, the existence of a solution set, the distinction between regression problems and classification problems, computational load, and other factors. For this purpose, a cheat sheet of machine learning algorithms can be used (see P. Sivasothy et al: "Proof of concept": "Machine Learning based filling level estimation for bulk solid silos"; Proc. Mtgs. Acoust.; Vol. 35; 055002; 2018). In the present invention, the machine learning algorithm may be (i) a deep learning algorithm such as a long short-term memory (LSTM) algorithm, a gated recurrent unit (GRU) algorithm, or a perceptron algorithm, (ii) an instance-based algorithm such as a support vector machine (SVM), particularly a deep learning algorithm. "Deep learning" may represent a method based on an artificial neural network (ANN) having a number of layers without boundaries of a certain size, which enables practical applications and optimized implementations while maintaining theoretical universality under mild conditions. Deep learning architectures for implementing deep learning algorithms include deep neural networks, deep belief networks (DBNs), recurrent neural networks (RNNs), convolutional neural networks (CNNs), and the like. Suitable optimization algorithms for manipulating the parameters of the learning algorithm during learning are known in the art and include, for example, gradient descent, momentum, rmsprop, Newton-based optimizers, adam, BFGS, or model-specific methods. These optimization algorithms are used during the training of the machine learning algorithm and modify the parameters at each training step so that the difference between the output of the machine learning algorithm and the expected output decreases until a predefined termination criterion such as the number of iterations or accuracy is obtained.

[0052] The determination of the trained machine learning model regarding the presence of the masking material may be provided to the display device for display on the screen. For example, a 3D color model (s) including a classification of which surface areas of each spatial substrate contain the masking material(s) may be provided to the display device for display on the screen, e.g., within a graphical user interface. Thereby, the user can confirm whether the presence and position of the masking material are correctly detected by the learned machine learning model, and can correct the presence and position if the masking material present on the spatial substrate is not correctly determined. Not only the result of the determination, but also the data related to the user's approval or correction(s) may be provided as a training data set to the computing device used to train the machine learning model via the communication interface. Thereby, the performance of the trained machine learning model can be improved using the data obtained during the method of the present invention.

[0053] Step (c): In step (c) of the method of the present invention, at least one computer processor is used to retrieve various data via a communication interface, namely, coating tool parameter data, coating procedure data, and substrate type data. The said data may be stored in a data storage medium such as an internal memory or a database connected to at least one computer processor via a communication interface. For example, it may be retrieved using the substrate classification data and coating material ID included in the provided spatial substrate data and coating material data. "Data storage medium" may represent a physical and other computer-readable medium for carrying or storing computer-executable instructions and / or data structures. Such a computer-readable medium may be any available medium accessible by a general-purpose or special-purpose computer system. The computer-readable medium may include a physical storage medium for storing computer-executable instructions and / or data structures. The physical storage medium includes computer hardware such as RAM, ROM, EEPROM, solid state drive ("SSD"), flash memory, phase change memory ("PCM"), optical disk storage devices, magnetic disk storage devices, or other magnetic storage devices, or any other hardware storage device(s) that can be used to store program code in the form of computer-executable instructions or data structures, which can be accessed and executed by a general-purpose or special-purpose computer system to implement the disclosed functions of the present invention. "Database" may represent a collection of related information that can be searched and retrieved. The database can be a searchable electronic numerical, alphanumeric, or text document, a searchable PDF document, a Microsoft Excel(R) spreadsheet, or a database generally known in the art. The database can be a set of electronic documents, photos, images, diagrams, data, or drawings existing in a computer-readable storage medium that can be searched and retrieved. The database can be a single database, a set of related databases, or a group of unrelated databases."Related database" means that there is at least one common information element in the related databases and such databases can be used to associate them.

[0054] Parameter data of the coating tool: The coating tool parameter data includes tolerance data of the coating tool and at least one application parameter related to the coating tool. The term "coating parameter related to the coating tool" represents a parameter used to apply a specific liquid or solid coating material using a defined coating tool. For example, when the coating tool is a spray applicator and the coating material is a liquid-based coating material, the parameter represents the spray parameters required to apply a specific liquid-based coating material to the surface of a spatial substrate using the spray applicator.

[0055] In one aspect, the tolerance data of the coating tool includes target distance data, overlap ratio data, pattern size data, rotational tolerance(s) related to the z-axis of the coating tool, rotational tolerance(s) related to the x-axis of the coating tool, rotational tolerance(s) related to the y-axis of the coating tool, or a combination thereof. The target distance data and overlap ratio may represent a defined target distance value / overlap ratio and / or a range of target distance / range of overlap ratio. By combining the defined values and ranges, a defined value can be used as a starting point for generating tool path data, and the starting point can be adjusted within the provided range if necessary to generate tool path data having the required quality. The term "target distance" represents the distance from the end of a coating tool, such as the nozzle of a coating material application device, to the surface of a spatial substrate, and may range from 6 inches to 12 inches. The pattern size may be fixed for a certain target distance or may be scaled according to the target distance. For example, a defined pattern size may be associated with a target distance of 6 inches and a target distance of 12 inches, and linear interpolation may be used to determine the pattern size for target distances within the range. Thus, the pattern size data may be composed of rules associating a given target distance with a specific pattern size or range of pattern sizes.

[0056] In one aspect, at least one application parameter related to the coating tool includes the flow rate of the coating material, the voltage applied to the coating material, the pressure applied to the coating material, the bell speed of the coating tool, or a combination thereof. The application parameters included in the coating tool parameter data depend on the type of coating tool, and if the coating tool is different, the application parameters included in the coating tool parameter data are also different. For example, an electrostatic spray applicator requires a voltage parameter, while a pneumatic spray applicator does not require such a parameter.

[0057] Coating tool parameter data is retrieved using at least one computer processor based on the provided coating material data. Retrieving coating tool parameter data based on the provided coating material data may include retrieving data indicating at least one type of coating material included in the provided coating material data, and optionally retrieving data indicating a coating tool, and retrieving coating tool parameter data associated with the retrieved data indicating at least one type of coating material and, optionally, the retrieved data indicating a coating tool. The coating tool parameter data may be stored in a data storage medium and may be mutually associated with data indicating the type of coating material such as the aforementioned unique ID. The unique ID may be used to retrieve appropriate coating tool parameter data from a data storage medium such as an internal memory or a database connected to at least one computer processor via a communication interface.

[0058] Data of coating procedure: The retrieved coating procedure data includes a set of rules for coating outer edges and edges adjacent to open spaces, and a set of rules for coating the main surface. The term "set of rules" represents an execution unit of rules and / or decision tables and / or algorithms, that is, the set of rules arranges rules and / or decision tables and / or algorithms into an execution unit. Multiple sets of rules can be executed in a defined order (also referred to as a rule flow). In a set of rules, rule priorities can be used to specify the execution order of the rules included in the set of rules. Each set of rules includes one or more rules. Each rule includes a condition and an action associated with that condition, that is, an action to be executed when the condition is met or determined to be true.

[0059] Each rule set may include one or more rules or algorithms. For example, a rule set for coating the outer edges of a spatial substrate and the edges adjacent to the open space(s) may include at least one algorithm for determining the outer edges and at least one algorithm for determining the edges adjacent to the open space(s) present within the surface spatial substrate. The algorithm for determining the edges adjacent to the open space(s) may determine the open space(s) that exist together with the spatial substrate that is at least partially surrounded by the determined edges adjacent to the open space(s). The "edge adjacent to the open space" refers to the edge that exists adjacent to the open space within the surface of the spatial substrate. Such open spaces are represented, for example, by marker light pockets, window frames, headlight pockets, etc. The "open space(s)" refers to holes present inside the spatial substrate, such as the marker light pocket of the fender, the window of the door, the hood scoop pocket of the hood, or depressions present inside the spatial substrate, such as the front light pocket of the fender, the upper inner rail and tabs of the fender, or the lower flange on the dog leg of the fender.

[0060] In one aspect, a rule set for coating an outer edge and an edge adjacent to an open space includes a rule for coating an edge adjacent to the open space, a rule for coating the outer edge, and optionally, a rule for coating an open space present within a spatial substrate that is at least partially surrounded by an edge adjacent to the open space. At least the rule for coating an edge adjacent to the open space and the rule for coating an open space present within the spatial substrate may include coating tool tolerance data different from the coating tool tolerance data included in the retrieved coating tool parameter data. In one example, the rule for coating the outer edge may also include coating tool tolerance data different from the coating tool tolerance data included in the retrieved coating tool parameter data. This may be the case, for example, for a spatial substrate that includes a grill such as an automobile bumper cover. Accordingly, the target distance data, overlap rate data, pattern size data, and rotation tolerance(s) included in the rules are different from the target distance data, overlap rate data, pattern size data, and rotation tolerance(s) included in the coating tool parameter data. Thereby, when an edge adjacent to an open space is detected and the open space is also detected, the retrieved coating tool tolerance data can be modified and different target distances and / or overlap rates and / or pattern sizes and / or rotation tolerances can be used for coating the open space and its edge. For example, for a coating tool to coat an open space and / or an edge adjacent to the open space, an angular variation of up to 45° from parallel to the surface plane may be allowed in the X-axis (i.e., pitch).

[0061] In one aspect, a rule set for coating a main surface includes rules for determining the start of a coating procedure, rules regarding the coating direction, rules regarding the rotation of a coating tool within a tool path, rules regarding the separation of surfaces, or combinations thereof. The "main surface" refers to the surface existing between the outer edges of a spatial substrate, excluding the open space on that surface. Thus, the main surface may or may not include open space, provided that the open space existing within the main surface is not counted towards the main surface.

[0062] By using the rule set during the generation of robot path data, such robot path data can be reliably generated for substrates having a high level of geometric variation. Thereby, using the generated robot path data, such substrates can be coated by a robot system, and coatings having high optical and mechanical quality can be obtained regardless of the geometric shape of each substrate. Thus, to ensure that the shape of the substrate is adequately considered during the generation of the robot path data and to avoid the application of inappropriate coating materials that may result in a decrease in the optical and / or mechanical quality of the resulting coating, such rule sets can be used to adjust the robot path data to each substrate without the need for manual operation. When at least two different coating materials are applied, the aforementioned coating tool parameter data and coating procedure data are retrieved for each coating material, and tool path data is generated for each coating material using the retrieved coating tool parameter data, coating procedure data, and retrieved spatial substrate data, as described below.

[0063] Data on substrate type: The substrate type data is retrieved based on the provided spatial substrate data, particularly based on the substrate classification data included in the provided spatial substrate data. The retrieved substrate type data includes at least one rule set regarding the type of spatial substrate that matches the substrate classification data included in the provided spatial substrate data. The term "type of spatial substrate that matches the substrate classification data" means that the type of the spatial substrate is the same as the aforementioned substrate classification data, or is the type of the spatial substrate associated with a unique classification ID. For example, when the substrate classification data is equal to "fender", the type of the spatial substrate is also "fender". Similarly, when the substrate classification data is equal to a unique classification ID, the type of the spatial substrate corresponds to the type associated with the said unique ID. When different types of spatial substrates are coated, the substrate type data is retrieved for each piece of spatial substrate classification data included in the spatial substrate data provided for each spatial substrate in step (a).

[0064] In one aspect, at least one rule set includes, for each type of spatial substrate, at least one rule for coating an edge adjacent to an open space (s), data regarding the required quality of a tool path (s), optionally at least one rule for coating an open space (s) present within the spatial substrate, and optionally at least one rotational tolerance of the coating tool. The rotational tolerance of the coating tool may include the aforementioned rotational tolerances regarding the x-axis, y-axis, and z-axis of the coating tool.

[0065] Retrieving the substrate type data may include determining the substrate classification data included in the provided spatial substrate data and retrieving the substrate type data associated with the determined substrate classification data from a data storage medium such as an internal memory or a database connected to at least one computer processor via a communication interface. For this purpose, the substrate type data may be associated with the substrate classification data before storing the data in a data storage medium such as an internal memory or a database connected to at least one computer processor via a communication interface.

[0066] Step (d): In step (d) of the method of the present invention, tool path data for the tool path to be traced by a coating tool along the surface of each spatial substrate is generated by at least one computer processor based on the data retrieved in step (c) and, optionally, the result of the determination performed in step (b). "Tool path" refers to the movement path of a coating tool relative to the surface of each spatial substrate, particularly a coating material applicator such as a spray applicator, and more particularly the relative movement path of the coating material applied from a coating tool such as a coating material applicator to the surface of each spatial substrate to be coated. Accordingly, the tool path includes the three-dimensional position of the distal end along the topographic profile of each spatial substrate offset therefrom and the locus of the orientation angle of the operating axis to avoid collision of the coating tool with each spatial substrate.

[0067] In one aspect, generating tool path data for the tool path to be traced by a coating tool along the surface of each spatial substrate includes the following: - Generating, by at least one computer processor, a 3D model of each spatial substrate based on the provided spatial substrate data and, optionally, applying a rule set to smooth the surface of each generated 3D model; - Determining, by a computer processor, outer edges, edges adjacent to open spaces, main surfaces, and open spaces present within each spatial substrate based on the retrieved coating procedure data and the generated and optionally smoothed 3D model; - Generating, by at least one computer processor, tool path data for outer edges and tool path data for edges adjacent to open spaces based on the retrieved coating procedure data, the retrieved coating tool parameter data, the retrieved substrate type data, the determined outer edges, edges adjacent to open spaces, and open spaces, and the generated and optionally smoothed 3D model; - Generating tool path data for the main surface of each spatial substrate based on the retrieved coating procedure data, the retrieved coating tool parameter data, the determined main surface, and the generated and optionally smoothed 3D model by at least one computer processor; - Optionally, repeating the step for at least one additional coating material based on the retrieved coating procedure data and the retrieved coating tool parameter data.

[0068] In one example, the 3D (three-dimensional) model of each spatial substrate is generated by at least one computer processor using the 3D point cloud data included in the spatial sub-state data provided in step (a). Each 3D model can be generated from point cloud data using commonly available computer software such as the open-source library Open3D. Each generated model may be smoothed using at least one rule set. The rule set may include rules that enable determination of the surface of each 3D model that can be smoothed and the degree of smoothing associated with the surface. Preferably, the degree of smoothing is selected such that the information required to generate the tool path data is not lost after the smoothing operation. Since the quality of the resulting coating can be controlled by the degree of surface smoothing, i.e., the higher the degree of surface smoothing, the lower the overall quality in terms of the optical appearance of the coated surface, the overall optical appearance of the resulting surface can be controlled by the degree of surface smoothing. For surfaces that do not require high quality in terms of optical appearance, such as the front light pocket present within the fender, a higher degree of smoothing may be used for the generated 3D model than for surfaces that require high quality in terms of optical appearance, such as the portion of the fender that is visible after the coated fender is attached to the vehicle body. By smoothing each 3D model, the mesh of each 3D model obtained from the respective point cloud data is converted into a surface, so that, as described later, it can be processed more efficiently, for example, by a computer processor that generates tool path data, while generating tool path data for each spatial substrate and the coating material applied on the spatial substrate.

[0069] Generating tool path data for outer edges and tool path data for edges adjacent to open spaces based on the retrieved coating procedure data, the retrieved substrate type data, and the generated and optionally smoothed 3D model using at least one computer processor may include the following: - Based on the determined outer edge, the determined edge adjacent to the open space, the determined open space, the retrieved coating tool parameter data, the retrieved coating procedure data, the retrieved spatial substrate data, and the generated and optionally smoothed 3D model, determining the target distance, overlap rate, pattern size, and rotation tolerance of the coating tool; - Using the generated and optionally smoothed 3D model to generate tool path data from the determined target distance, overlap rate, pattern size, and rotation tolerance of the coating tool.

[0070] Generating tool path data for the main surface based on at least one computer processor, the retrieved coating procedure data, in particular a set of rules for coating the main surface, the retrieved coating tool parameter data, and the generated and optionally smoothed 3D model may include the following: - Determining, based on the retrieved coating procedure data and the generated and optionally smoothed 3D model, whether the main surface of each spatial substrate includes at least two distinct surfaces; - Determining whether the main surface of each spatial substrate includes a convex shape and creating a hull around the spatial substrate(s) using the convex hull method; - Based on the determined separation surface, the determined convex shape, the retrieved coating tool parameter data, the retrieved coating procedure data, and the generated and optionally smoothed 3D model, determining the target distance, overlap rate, pattern size, and rotation tolerance of the coating tool; and - Using the generated and optionally smoothed 3D model to generate tool path data from the determined target distance, overlap rate, pattern size, and rotation tolerance of the coating tool.

[0071] In this embodiment, the presence of a separation surface is determined by determining whether each spatial substrate includes a surface having a certain relative angle and radius of curvature between the two resulting surfaces. For example, if the determined relative angle is from 25 to 90° and the radius of curvature is from 1 to 10 inches, surface separation is provided. By ensuring that both the range of the relative angle and the radius of curvature are satisfied, although the radius of curvature is small, a small body line with little difference in the angle between the two resulting surfaces can be reliably determined to be separate surfaces, thus avoiding unnecessary separation of the surface of the spatial substrate.

[0072] The outer skin around the convex surface of a spatial substrate, such as the convex surface of a bumper cover, can be determined, for example, using a generally known convex hull algorithm that can calculate the convex hull of a 3D spatial substrate. Then, the outer shell can be used to generate the tool path data of the surface.

[0073] In one embodiment, generating the tool path data of the tool path that the coating tool follows along the surface of each spatial substrate further includes calculating the respective coating material application using the generated tool path data and determining whether the coating obtained from the calculated coating material application meets at least one predetermined parameter. If the coating does not meet at least one preset parameter, at least one computer processor may use the determination result to optimize the generated tool path data. Thereby, the coating obtained as a result of applying the coating material using the robot path data generated from the tool path data can be made to meet the predefined quality parameters such as wet and / or dry film thickness, surface area to be coated, and the like.

[0074] The predefined parameters may include, for example, wet and / or dry film thickness or its range, and / or surface area to be coated or its range.

[0075] Calculation of coating material application using the generated tool path data may include simulation of coating material application using the generated tool path data and the specific applicator used for each coating layer, which may be, for example, by using a shader that simulates the coating coverage by projecting light onto the surface of a 3D model. By this simulation, a coverage map of the generated and optionally smoothed 3D model can be calculated using predefined parameters regarding how much material is deposited (vapor-deposited). The coverage map can be used to determine whether a specific target area of the 3D model of each spatial substrate is coated with an appropriate proportion of coating material such that the wet film thickness and / or dry film thickness of the resulting coating layer meets a predefined range. Also, the coverage map can be used to control the thickness of the transparent coating material and the thickness of the coating in parts of the spatial substrate that require specific specifications regarding transparency, such as in ADAS (Advanced Driver Assistance System) and radar sensors. The calculated coverage map may be provided via a communication interface, for example, to a display device and provided for display on the device. The coverage map may also be displayed within a graphical user interface (GUI), similar to the further calculated data. The coverage map may include different colors to indicate whether predefined parameters are met.

[0076] Step (e): In step (e) of the method of the present invention, at least one computer processor generates robot path data based on the tool path data generated in step (d).

[0077] The generation of the robot path may include determining the collision geometry within the working space of the robot based on the spatial substrate data, and determining the robot path data based on the determined collision geometry and the generated tool path data. The determination of the collision geometry ensures that the robot does not collide with and destroy each spatial substrate during the coating procedure. The robot path data is generated such that the robot does not collide with each spatial substrate existing within its workspace and can follow the generated tool path data - using a coating tool. This may include determining the initial pose of the robot, determining all subsequent robot movements to follow the generated tool path data, and determining the return to the initial pose.

[0078] In one aspect, the generation of the robot path data further includes: - Using at least one computer processor to sort the generated tool path data so that the robot path generated from the tool path data for the outer edges and the edges adjacent to the open space is executed before or after the robot path generated from the tool path data for the main surface, and / or - Optimizing the generated or sorted robot path data by at least one computer processor.

[0079] Prior to the robot path(s) for the main surface, executing the robot path(s) for the outer edge portion and the edge portion adjacent to the open space causes overspray of the coating on the edge portion onto the main surface, but this overspray can be covered by subsequently coating the main surface so as to avoid an adverse effect on the final overall appearance, and thus it can be beneficial. Further, this avoids applying too much coating material to the edge, which is undesirable because if there is too much coating material on the edge, it is likely to sag, flow, and form a heavy edge, which has an adverse effect on the final overall appearance of the coated substrate. However, it is also possible to first execute the robot path (robot path) for the main surface and then execute the robot path for the edge. In this case, it is necessary to adapt the application parameters so that the overspray generated by the coating of the edge portion is hidden and does not adversely affect the overall appearance.

[0080] Optimization of the generated or sorted robot path data can be beneficial because it can smooth the movement of the robot and make it more efficient and consistent. Optimization of the generated or sorted robot path data can be performed based on open source libraries such as Descartes (a ROS-Industrial project for performing path planning on a poorly defined Cartesian trajectory) and software frameworks such as trajopt (a software framework for generating robot trajectories by local optimization).

[0081] Step (f): In step (f) of the method of the present invention, the generated robot path data is provided via a communication interface. If the robot path data is sorted and / or optimized, the sorted or optimized robot path data is provided via the communication interface.

[0082] In one example, this includes providing the generated robot path data to a robot controller connected via a communication interface to a robot constituting the coating tool, as will be described later.

[0083] In another example, this includes providing the generated robot path data to a display device and causing it to be displayed thereon. This may include a simulation of the robot path data for visualizing the movement of the robot such that the user can verify that the determined robot path data will not cause an obvious breakage of the spatial substrate or the robot during the coating process. The user may have to approve the generated robot path data before providing it to the robot controller.

[0084] In yet another example, the generated robot path data is provided to a data storage medium via a communication interface. For this purpose, the generated robot path data may be mutually associated with a spatial substrate ID so that the robot path data generated when the same spatial substrate is coated again can be retrieved, as a result of which the time required to generate the required robot path data is shortened.

[0085] Further step: Apart from steps (a) to (f), the method of the present invention may further include additional steps. In one aspect, the method of the present invention further includes a step of determining, by at least one computer processor, the amount of each coating material required to coat each spatial substrate based on the generated tool path data and the provided coating material data. The amount of each coating material can be calculated from the flow rate included in the retrieved coating tool parameter data and the generated tool path data. In particular, the data indicates the coating duration, that is, data related to the duration during which the coating tool is turned on during the coating process of each spatial substrate. This additional step may be executed before step (e), before step (f), or after step (f). The determined amount of each coating material may be provided via a communication interface to a display device for display, for example, within a GUI, and / or to a mixer that enables automatic preparation of the amount of each coating material. By executing this additional step, the user can obtain information regarding the amount of coating material required for the coating process. Thus, this data can be used for inventory planning or to determine the amount of coating material that needs to be prepared for the coating procedure of each spatial substrate. In the latter case, since there is no need to prepare more paint than required for the painting process, waste and costs associated with wasted paint can be reduced. Further, the determined amount and information regarding the paint included in the paint data such as the paint ID are provided to an automatic mixer, and the automatic mixer can mix the amount determined based on the received data. Thereby, the coating process can be fully automated, and waste of the coating material due to mixing errors can be prevented.

[0086] At least one computer processor that executes steps (d) to (f) may be the same computer processor or different computer processors, that is, the computer processor that executes step (d) may be different from the computer processors that execute steps (e) and (f). In one example, the computer processors that execute steps (d) to (f) may be arranged on a server such that steps (d) to (f) are executed in a cloud computing environment, or may be connected to a further computing device that functions as a client device. A "client device" may represent a computer or program that depends on sending requests to other programs as part of its operation, or the hardware or software of a computer that accesses services made available by a server. Preferably, the server is an HTTP server and this is accessed using conventional Internet web-based technology. The client device provides spatial substrate data to the server via a communication interface, similar to the data retrieved in step (c). Thereby, since the calculations that consume a large amount of resources are executed on the server, a client device with low computing power can be used. An Internet-based system is particularly useful when a service for generating robot path data is provided to customers via the Internet.

[0087] Embodiment of the computing device of the present invention In one aspect, the computing device of the present invention further comprises at least one database containing spatial substrate data, coating parameter data, coating procedure data, substrate type data, coating material data, or combinations thereof.

[0088] In one embodiment, the aforementioned data may be stored in a single database. In another example, at least a part of the spatial substrate data, coating parameter data, coating procedure data, substrate type data, and coating material data may be stored in the same database, that is, the database may include, for example, the spatial substrate data and the coating material data. In still another example, the aforementioned data may be stored in separate databases respectively.

[0089] Embodiments of the robot system of the present invention The robot system may include a computing device of the present invention for generating robot path data, and a robot device configured to receive the generated robot path data and apply at least one coating material to at least a part of the surface of each spatial substrate using the received robot path data. The robot system may include a computing device for generating robot path data, including at least one processor and a memory storing instructions that configure the device to perform the steps of the methods disclosed herein when executed by the processor.

[0090] In one aspect, the robot device includes at least one movable robot member including a coating tool, and a robot control device adapted to receive the robot path data generated from the device and operable to move the at least one movable robot member according to the received robot path data to coat at least a part of the surface of each spatial substrate with at least one coating material. The coating tool is preferably a coating material applicator such as a spray applicator generally used for applying a liquid or solid coating material to at least a part of the surface of the substrate. The last movable robot member may be a movable robot arm configured to include a coating tool.

[0091] In one aspect, the robot device further comprises a sensor system configured to generate data on the working space of the robot and / or data indicating the shape and color of each spatial substrate, and to provide the generated data to the computing device of the present invention. The sensor system is preferably also used to acquire data on the shape and color of each spatial substrate based on the scan path data provided to the robot controller. The term "comprises" includes not only permanently attached sensor systems but also temporarily attached sensor systems. The sensor system is preferably attached to at least one movable robot member of the robot device. In one embodiment, the robot device includes a movable robot member including a coating tool, similar to the sensor system. In another embodiment, the robot device includes a first movable robot member including an application tool and a second movable robot member including the sensor system. In yet another embodiment, the robot device includes only one movable robot member including either an application tool or a sensor system, i.e., the application tool and the sensor system are only temporarily attached to the movable robot member. The temporary attachment of the sensor system is beneficial to avoid the application of coating materials on the sensor system that may lead to performance degradation or failure of the sensor system. The switching between the coating tool and the sensor system may be performed using a tool changer as described later.

[0092] The sensor system may include a depth sensor, and the sensor system may be selected from either a camera system such as a laser scanner commercially available from Framos GmbH and a laser scanner. The camera system or the laser scanner must be able to acquire data on the working space of the robot and / or the shape of the spatial substrate, such as color data and 3D point cloud data.

[0093] In one aspect, a robot system comprises a plurality of coating tools, each coating tool including at least one coating material reservoir containing a different coating material, which is connectable to a robot device, particularly a movable robot member. The robot device is preferably configured to select an appropriate coating tool from the plurality of coating tools based on coating material data received from a computing device. The coating tool may include a code, barcode or unique ID that enables the robot device to identify the coating tool to which it is attached, and the robot device may be configured to identify the appropriate coating tool based on the code, barcode or unique ID attached to the tool and attach the appropriate coating tool to the robot device, particularly the movable robot member.

[0094] The robot system is preferably installed in a spray booth which further includes heating means. The heating means may be used to dry and / or cure the applied coating material to form a cured coating on each spatial substrate. In contrast to the dry coating film which is still sticky and undergoes further property changes by the use of curing conditions (such as heat), the cured coating film is a solid coating film that does not undergo further property changes even when exposed to curing conditions. The heating means may be controlled by a computing device of the system of the present invention which is configured to search for the drying and / or curing temperature and drying and / or curing time by using, for example, the coating material type ID or coating material ID included in the data, based on the provided coating material data, to search for the appropriate drying and / or curing temperature(s) from a database and provide the searched temperature(s) and time to the heating means. The heating means can be turned on after the robot control device provides a signal indicating the end of the coating procedure or the end of the application of a specific coating material to the computing device, for example, after the last coating material has been applied to each spatial substrate and the robot device has returned to its initial position, or after the robot device has returned to the tool rack to replace the current coating tool with another coating tool made of a different coating material. Thereby, the coating and curing processes can be fully automated, thus minimizing human operation and providing a reproducible coating procedure despite the wide variety of substrate shapes.

Brief Description of the Drawings

[0095] These and other features of the present invention are described more fully in the following description of exemplary embodiments of the present invention. To easily identify the discussion of any particular element or act, the leading digit or digits of the reference number refer to the figure number in which the element is first introduced. This specification is presented with reference to the accompanying drawings:

Figure 1

Figure 2

Figure 3AB

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12A

Figure 12B

[0096] [Detailed Description] The detailed description set forth below is intended to be a description of various aspects of the subject matter and is not intended to represent the only configuration in which the subject matter may be implemented. The accompanying drawings are incorporated herein and constitute a part of the detailed description. The detailed description includes specific details for a thorough understanding of the subject matter. However, it will be apparent to those skilled in the art that the subject matter may be practiced without these specific details.

[0097] In some cases, the illustration of separating various components in the figures into separate units may reflect the use of corresponding separate physical and tangible components in an actual implementation. Alternatively, or in addition thereto, any single component illustrated in the figures may be implemented by a plurality of actual physical components. Or, alternatively or in addition thereto, the depiction of two or more separate components in the figures may reflect different functions performed by one actual physical component.

[0098] In other figures, the concepts are described in the form of flowcharts. In this format, specific operations are described as constituting separate blocks that are executed in a certain order. Such embodiments are illustrative and non-limiting. The specific blocks described herein can be grouped together and executed in a single operation, the specific blocks can be broken down into multiple component blocks, and the specific blocks can be executed in an order different from that illustrated herein (including modes of executing the blocks in parallel). In one embodiment, the blocks shown in the flowchart related to the processing-related functions can be implemented by the hardware logic circuit described in relation to FIG. 6, and the hardware logic circuit can in turn be implemented by one or more hardware processors and / or other logic components including a set of task-specific logic gates.

[0099] Regarding terms, the phrase "configured to" includes various physical and tangible (specific) mechanisms for performing the specified operation. The above mechanisms can be configured to perform the operation using the hardware logic circuit described in relation to FIG. 6. The term "logic" similarly includes various physical and tangible mechanisms for performing a task. For example, each processing-related operation illustrated in the flowchart corresponds to a logic component for performing that operation. The logic component can perform that operation using the hardware logic circuit as described in relation to FIG. 6. When implemented by a computing device, the logic component represents an electrical component that is a physical part of the computing system, regardless of how it is implemented.

[0100] Any of the memory resources described in this specification, or combinations of memory resources, may be regarded as a computer-readable medium. In many cases, a computer-readable medium represents some physical and tangible entity. The term computer-readable medium also includes, for example, propagated signals that are transmitted or received via some physical conduit and / or air or other wireless medium. However, the specific term "computer-readable storage medium" explicitly excludes the propagated signal itself, while including all other forms of computer-readable media.

[0101] In the following description, one or more functions may be specified as "optional". This type of description should not be construed as an exhaustive listing of features that may be considered optional. That is, other features may also be considered optional even if not explicitly specified in the text. Further, a description of a single entity is not intended to exclude the use of such multiple entities. Similarly, a description of multiple entities is not intended to exclude the use of a single entity. Further, in this specification, specific features may be described as alternative ways of performing a specified function or implementing a specified mechanism, while the features may also be combined together in any combination. Finally, the terms "exemplary" or "illustrative" represent one implementation among potentially many implementations.

[0102] FIG. 1 shows a flow diagram of a first non-limiting embodiment of a method 100 for generating robot path data for a robot path (s) that a robot including a coating tool follows while coating at least a portion of the surface of at least one spatial substrate with at least one coating material, the method being implemented by a computing device including a computer processor. The method 100 may be used to coat at least a portion of the surface of a spatial substrate with a coating material using a robot system including a robot including a coating tool. The computing device may be a computing system described in connection with FIG. 6 below, preferably a stationary device such as a stationary computer including a computer monitor, or a mobile computing device, or may be located in a cloud computing environment. The spatial substrate to be coated may be an automotive part, such as an automotive body part, such as an automotive fender. The spatial substrate to be coated may be an automotive hood, an automotive door, and an automotive bumper cover, or other automotive spatial substrate. A plurality of spatial substrates may be coated, and they may be of the same substrate type or different substrate types. Thus, a plurality of spatial substrates having various geometric shapes may be coated. An automotive fender may include a conversion layer but may not include a further coating layer. An automotive fender may already include at least one primer layer, a primer-surface treatment layer, or a primer layer. A colored basecoat material and a clearcoat material may be applied by a robot constituting the coating tool. The coating material applied by the robot may be more or less. The robot may be arranged, for example, in a spray booth as described in connection with FIGS. 7 to 11 below.

[0103] In block 102, the spatial substrate data and coating material data of each spatial substrate may be provided to a computing device that implements method 100. The provided spatial substrate data may at least include or contain substrate classification data and data indicating the shape and color of the surface of the spatial substrate, particularly the spatial substrate. In this example, the spatial substrate data includes data representing the spatial substrate in 3D space, particularly 3D point cloud data of the spatial substrate, as well as color data of the spatial substrate, particularly RGB data or still image data acquired by a scanning device (or sensor system) described in relation to FIG. 7 later. In this example, the spatial substrate data further includes data indicating the identity of the substrate, such as the substrate ID, substrate name, barcode, QR code, etc., and a rule set for smoothing the surface of the 3D model generated from the data indicating the shape and color of the spatial substrate.

[0104] The spatial substrate data may be provided as described in relation to FIG. 2 below. The spatial substrate data may be retrieved from a data storage medium such as an internal storage device of the computing device implementing method 100 or a database connected via a communication interface to the computing device. The stored spatial substrate data may be associated with the substrate ID so that the spatial substrate data can be retrieved using the substrate ID. The substrate ID may be input by the user via a graphical user interface that enables the user to input the substrate ID or select the substrate ID from a list of available substrate IDs for the spatial substrate data.

[0105] Coating material data includes data indicating the identification of the coating material. The data indicating the identity of the coating material may include the type of each coating material, i.e., the name of the base coat and the clear coat. The data indicating the identification of the coating material may include the ID of each coating material type. Providing coating material data may include displaying, by a computing device, a user interface that enables a user to select a coating material type from a displayed list of available types. The user interface may enable the user to input the ID or name of each coating material type. The user interface may enable the user to provide the ID / barcode / QR code of the coating material to be applied by a robotic coating tool, and the computing device may obtain, based on the provided ID / barcode / QR code, data indicating the identity of the coating material from a data storage medium.

[0106] Coating material data may further include data indicating the identification of the coating material, such as ID, barcode, QR code, etc., property data of the coating material, such as chemical and / or physical property data, data regarding the composition of each coating material, or a combination thereof. Coating material data may further include data indicating the spatial substrate to be coated with the coating material(s). This may be particularly preferred to ensure that the correct coating material is applied to each spatial substrate when multiple spatial substrates are coated with different coating materials. The property data and the data regarding the composition may be stored in a data storage medium and may be retrieved by a computing device using the provided data indicating the identity of the coating material.

[0107] The order of the base coat and the clear coat applied by the robot's coating tool does not have to be provided by the user, but may be determined by the computing device using a rule set stored in a data storage medium connected to the computing device. The order of the base coat and the clear coat applied by the robot's coating tool does not have to be provided by displaying a user interface that prompts the user to select the order of the types of coating materials provided.

[0108] Data indicating the coating tool does not have to be provided by the user, but may be retrieved from the data storage medium based on the coating material type or coating material ID provided. Data indicating the coating tool may be provided, for example, by displaying a user interface that prompts the user to select the coating tool by displaying a list of available coating tools.

[0109] In block 104, method 100 may determine whether the surface of the spatial substrate includes at least one masking material based on the spatial substrate data provided in block 102, and this step is generally optional. Block 104 may exclude the areas covered by the masking material when determining the tool path data and thus the robot path data, in order to avoid applying the coating material to the masking material, so as to reduce the consumption of the applied coating material and avoid overspray, and may be executed. Block 104 may be executed by a computing device that executes other blocks of method 100, or may be executed by a further computing device such as a server device that is connected to the computing device via a communication interface and has access to a trained machine learning model used to determine the presence of the masking material on the spatial substrate. In this case, the computing device may function as a client device and may provide the spatial substrate data to the server device. Next, the server device may use the provided spatial substrate data and the learned machine learning model to determine the presence of the masking material and provide the determination result to the computing device. Using a further server device to determine the presence of the masking material may be beneficial because the trained machine learning model can be stored in a database accessible only to the server, which can reduce the complexity of the overall system. Furthermore, the user feedback regarding the correct determination of the masking material may also be used by the server device to improve the trained machine learning model using generally known learning techniques.

[0110] Determining whether a spatial substrate includes at least one masking material based on provided spatial substrate data may include generating a three-dimensional (3D) model using the provided spatial substrate data and providing the generated 3D model to a trained machine learning model, particularly a deep learning algorithm. The 3D model may be created using 3D point cloud data and color data included in the provided spatial substrate data, i.e., the 3D model is a 3D color model. Generation of the 3D model from the point cloud data and color data may be performed according to methods well known in the art.

[0111] The machine learning model may be trained with past data of the masking material, particularly past 3D color models of spatial substrates including the masking material(s), to determine the presence of the masking material(s) on the surface of the spatial substrate based on the provided past spatial substrate data. The machine learning model, particularly a deep learning algorithm, may be hosted by a computing device, a remote server, or a cloud or other server implementing method 100. Advantageously, by placing the algorithm on a remote server or cloud server, the cost of additional memory and / or a more complex processor when using the algorithm to determine the presence of the masking material(s) on the surface of each spatial substrate can be avoided. Further, continuous or periodic improvement of the algorithm can be more easily performed on a centrally located server, avoiding the data cost and risk of pushing out algorithm updates to each computing device. The remote server may also serve as a central repository (vault) for storing a collection of training and / or operation data transmitted from various computing devices for use in training and developing existing algorithms. For example, the growing repository of data can be used to update and improve algorithms on existing systems and provide improved algorithms for future use.

[0112] The trained machine learning algorithm used in block 104, and more specifically the artificial neural network (ANN) model, may be obtained using generally known machine learning methods in order to train the ANN model to determine the presence of masking material on the 3D color model of the spatial substrate. An exemplary commercially available software for implementing the training process is Keras (available on the Internet at Keras.io), which is an open-source ANN model library that runs on top of either TensorFlow or Theano, which provides the necessary computational engine. TENSORFLOW (a trademark of Google, Inc., Mountain View, California) is an open-source software library originally developed by Google, Inc., Mountain View, California, and is available as an Internet resource at www.twnsirflow.org.

[0113] The model training dataset used for training a machine learning model can be divided into three parts: a training set, a validation set, and a verification (or "test") set. The training set is used to adjust the internal weighting algorithms and functions of the hidden layers of a neural network so that the neural network can iteratively "learn" how to correctly recognize and classify patterns in the input data. However, the validation set is mainly used to minimize overfitting. The validation set usually does not adjust the internal weighting algorithm of the neural network like the training set, but rather verifies that an improvement in accuracy for the training dataset leads to an improvement in accuracy for a dataset that has not previously been applied to the neural network, or at least a dataset for which the network has not yet been trained (i.e., the validation dataset). If the accuracy for the training dataset improves but the accuracy for the validation dataset remains unchanged or decreases, the process is often called "overfitting" of the neural network, and training needs to be stopped. Finally, the validation set is used to test the final solution to confirm the actual predictive power of the neural network.

[0114] In one example, approximately 70% of the developed or collected data model set is used for model training, 15% is used for model validation, and 15% is used for model verification. These approximate divisions can be changed as needed to obtain the desired results. For example, about 300 sets of data can be collected, and each set contains a 3D model of a spatial substrate with a masking material and a 3D model of a spatial substrate without a masking material. The training dataset may include samples covering the entire expected range of spatial substrates and the position and type of the masking material on the substrate.

[0115] The result of the determination may be provided to the computing device, and the computing device may display the result of the determination on a graphical user interface so that the user can confirm whether the masking material present on the spatial substrate has been correctly recognized. The provided result may represent an overlay of the processed 3D model by the machine learning model onto the unprocessed 3D model (i.e., the 3D model generated from the spatial substrate data). A 3D color model including a classification of which surface regions include the masking material(s) may be provided to a display device, for example, for display on the screen within a graphical user interface. Thereby, the user can confirm whether the presence and position of the masking material have been correctly detected by the learned machine learning model, and can correct the presence and position if the masking material present on the spatial substrate has not been correctly determined.

[0116] In block 106, the computing device implementing method 100 may search for coating tool parameter data based on the coating material data provided in block 102. If more than one coating material is to be applied, i.e., if the coating material data provided in block 102 includes data for two or more coating material types, the coating tool parameter data associated with each coating material type included in the provided coating material data may be searched in block 106. The coating tool parameter data may include coating tool tolerance data and at least one application parameter associated with a coating tool used by a robot to coat at least a portion of the surface of a spatial substrate with each respective coating material. The coating tool parameter data may be stored in a database connected to the computing device via a communication interface, and data included in the coating material data provided in block 102, such as the name or ID of the type of coating material to be applied, may be used by the computing device to search for the appropriate coating tool parameter data. Searching for coating tool parameter data based on the coating material data provided in block 102 may include searching for data indicating the types of coating materials (i.e., base coat and clear coat) included in the provided coating material data, and searching for coating tool parameter data associated with the base coat and clear coat.

[0117] The tolerance data of the coating tool can include target distance data, overlap rate data, pattern size data, rotational tolerances (plural possible) regarding the z-axis of the coating tool, rotational tolerances (plural possible) regarding the x-axis of the coating tool, and rotational tolerances (plural possible) regarding the y-axis of the coating tool. The target distance data and the overlap rate can represent a defined target distance value / overlap rate and / or a range of target distance / overlap rate. For example, the target distance data includes a range of 4 to 8 inches, and the overlap percentage data includes a range of 70 to 90% in the case of the base coat material. In the case of the clear coat material, the target distance data may include a range of 6 to 10 inches, and the overlap rate data may include a range of 40 to 60%. The target distance data and the overlap rate data may each include a defined value such as 6 inches for the base coat material and 8 inches for the clear coat material, and an allowable range to enable generation of appropriate tool path data. The combination of the defined value and the range allows for using the defined value as a starting point for generating the tool path data and, if necessary to generate tool path data with the required quality, adapting the starting point within the provided range. The overlap rate data may remain fixed while other parameters may be adjusted during the generation of the tool path data. The overlap rate data may be adjusted during the generation of the tool path data. The pattern size data may be fixed for a certain target distance or may be scaled according to the target distance. For example, assuming a pattern with a height of 10 inches and a width of 1.5 inches for a target distance of 6 inches and a pattern with a height of 14 inches and a width of 2.5 inches for a target distance of 12 inches, the pattern size data between these two points can be obtained using linear interpolation between these two points. The pattern size data may include rules correlating a given target distance with a specific pattern size or a pattern size range.The rotational tolerances for the z-axis (roll) and y-axis (yaw) of the coating tool may be within 15° of the normal surface vector, and the rotational tolerance for the x-axis (pitch) of the coating tool may be within 5° of the normal surface vector.

[0118] At least one coating parameter related to the coating tool may include the flow rate of the coating material, the voltage applied to the coating material, the pressure applied to the coating material, and the bell speed of the coating tool. This may be preferred when the coating tool corresponds to an ESTA (electrostatic spray coating) applicator. At least one coating parameter related to the coating tool may include the flow rate of the coating material and the pressure applied to the coating material. This may be preferred when the coating tool corresponds to a pneumatic coating material applicator.

[0119] In block 108, the computing device implementing method 100 may search for coating procedure data (CPD). The retrieved coating procedure data may include a rule set for coating outer edges and edges adjacent to open space(s), and a rule set for coating the main surface. The rule set for coating outer edges and edges adjacent to open space(s) of the spatial substrate may include at least one algorithm for determining outer edges and at least one algorithm for determining edges adjacent to open space(s) present within the surface spatial substrate. The algorithm for determining edges adjacent to open space(s) may further enable determining open space(s) that exist with the spatial substrate at least partially surrounded by the determined edges adjacent to the open space(s). The rule set for coating outer edges and edges adjacent to open space(s) may include rules for coating edges adjacent to open space(s), rules for coating outer edges, and optionally rules for coating open space(s). At least, the coating rules for edges adjacent to open space(s) and the coating rules for open space(s) may include coating tool tolerance data different from the coating tool tolerance data included in the retrieved coating tool parameter data. Thereby, when an edge adjacent to open space(s), and thus the open space(s), is detected, the retrieved application tool tolerance data can be modified, and different target distances and / or overlap rates and / or pattern sizes and / or rotational tolerances can be used for coating the open space(s) and its edges as well as the outer edges.

[0120] A rule set for coating a main surface may include rules for determining the start of the coating procedure, rules regarding the coating direction, rules regarding the rotation of the coating tool within the tool path, and rules regarding the separation of surfaces. The rule set for coating the main surface can include more rules or fewer rules. The rule for determining the start of the coating procedure can include, for example, determining the corner of the substrate and proceeding in a direction that allows maintaining the vertical direction of the reservoir of the coating material of the coating tool. Examples of rules regarding the application direction include applying the substrate from top to bottom, or from bottom to top, and / or restricting the tool path with curvature for the end of the substrate (either not following the curvature of the tool path or only following the curvature up to a certain range). Rules regarding the rotation of the coating tool within the tool path include, for example, restricting the rotation so that the coating tool does not change its orientation during the coating of the surface of the spatial substrate (or target area). Rules regarding the separation of surfaces include determining the separated surface and coating the separated, i.e., adjacent surfaces. The presence of separate surfaces (or different target areas) may be determined by determining whether the spatial substrate includes surfaces having a relative angle between the resulting two surfaces and the radius of curvature. For example, if the determined relative angle is from 25° to 90° and the radius of curvature is from 1 to 10 inches, surface separation is given. By ensuring that both the range of the relative angle and the radius of curvature are satisfied, a small body line with a small radius of curvature but little difference in the angle between the resulting two surfaces is surely determined to be a separate surface, avoiding unnecessary separation of the surface of the spatial substrate. Coating separate surfaces may include coating adjacent surfaces in an adjacent-next pattern to maintain a wet film on both adjacent surfaces.As a result, the freshly coated surface is wet to accept overspray, and it is ensured that the adjacent surface to be coated has wet overspray when it is coated.

[0121] The retrieved coating procedure data may further include a set of rules for applying at least two different coating materials on the same spatial substrate. This may be preferred when the coating process enables applying multiple coating materials on the same spatial substrate, such as applying a clear coat material following a base coat material. The coating procedure data may, for example, not include the set of rules when the coating process enables applying only a single coating material on the same spatial substrate. The set of rules may include rules for copying tolerance data of the coating tool used for applying the previous coating material (in this case the base coat material) and modifying the copied data based on the coating material data provided in block 102. Further, the set of rules may include rules for removing or performing the determination of an open space (s) adjacent edges, and thus an open space (s). Thereby, it is possible to coat an open space (s) with a further coating material (in this example, a clear coat material), or to avoid coating an open space (s) with a further coating material if, for example, a clear coat layer is not required on an open space (s) coated with a base coat layer.

[0122] By using a rule set during the generation of robot path data, such robot path data can be reliably generated for substrates having a high level of geometric variation. Thereby, using the generated robot path data, such substrates can be coated by a robot system, and coatings having high optical and mechanical quality can be obtained regardless of the geometric shape of each substrate. Therefore, in order to ensure that the shape of the substrate is fully considered during the generation of the robot path data and to avoid the application of inappropriate coating materials that may reduce the optical and / or mechanical quality of the resulting coating, such a rule set can be used to adjust the robot path data to each substrate without the need for manual operation.

[0123] In block 110, the computing device implementing method 100 may search for substrate type data (STD) based on the spatial substrate data provided in block 102. For this purpose, at least one processor included in the computing device may determine the substrate classification data included in the spatial substrate data provided in block 102 and obtain the relevant substrate type data from, for example, a database or the internal memory of the computing device. The obtained substrate type data includes at least one rule for coating an edge adjacent to an open space for the corresponding type of spatial substrate, at least one rule for coating an open space present in the spatial substrate, and at least one rotational tolerance of the coating tool. The at least one rule set may include more rules or fewer rules, or may not include the rotational tolerance(s) of the coating tool.

[0124] In block 112, a computing device implementing method 100 may generate tool path data based on the retrieved coating tool parameter data, the retrieved coating procedure data, the retrieved substrate type data, and, if any block is executed, based on the determination result of block 104. The tool path data may be generated as described in FIGS. 3A and 3B below.

[0125] In block 114, a computing device implementing method 100 may generate robot path data based on the tool path data generated in block 112. Generating the robot path may include determining collision geometries within the robot's workspace based on spatial substrate data and determining robot path data based on the determined collision geometries and the generated tool path data. Determining the collision geometries ensures that the robot does not collide with and damage the spatial substrate during the coating procedure. The initial pose of the robot may be determined, and based on that initial pose, all subsequent robot movements may be generated such that the robot, particularly the movable robot member described in relation to FIG. 7, can follow the generated tool path data and return to the initial pose. The generated tool path data may be sorted before generating the robot path data. The generated robot path data may be optimized as described in relation to FIG. 4 below.

[0126] In block 116, the robot path data generated in block 114 may be provided via a communication interface. This may include providing the generated robot path data, as will be described in connection with FIG. 7 below, not only to a computing device but also to a robot controller connected via the communication interface to a robot. The generated robot path data may further be associated with the ID of the spatial substrate and may be provided to a data storage medium such as a database or internal memory. After the end of block 116, method 100 may end, or may return to block 102, for example, when it is detected that spatial substrate data and coating material data have been provided.

[0127] FIG. 2 shows a flowchart of method 200 for generating the spatial substrate data described in connection with block 102 of FIG. 1. The generated spatial substrate data may include a substrate ID and a rule set for smoothing the surface of a 3D model generated from data indicating the shape and color of each spatial substrate. Method 200 may be executed by the computing device described in connection with FIG. 1, and the computing device includes at least one processor that implements a routine for performing the steps described in connection with blocks 202 to 216 below.

[0128] In block 202, the routine implementing method 200 may detect a user input indicating a substrate classification associated with each spatial substrate and a user input indicating the position of each spatial substrate within the working space of the robot. The user input may be detected by displaying a user interface on a display of the computing system implementing method 100. The user interface may enable the user to select the substrate classification of each spatial substrate present within the working space of the robot, for example, by displaying a list of possible substrate classifications such as trunk, hood, fender, etc., or by displaying a text field and prompting the user to enter each substrate classification(s). Further, the user interface may enable the user to select the position of each spatial substrate within the working space of the robot, i.e., within the spray booth, for example, by displaying a map or an image of the spray booth and prompting the user to select the position of each spatial substrate within the displayed map of the spray booth. The spray booth may be divided into different zones such as five zones, and the user may be prompted to select all the zones containing the spatial substrates. The zones may be marked within the spray booth to facilitate the user's selection of the zones. The user input may be detected via an input device or an input / output device, particularly interaction elements such as a mouse, a keyboard, a trackball, a touch screen, or a combination thereof, and these devices may be attached to the computing device implementing method 200 via a communication interface.

[0129] In block 204, the substrate classification data and the positions of each spatial substrate within the working space of the robot may be determined based on the detected user input. When the user input is detected by a touch screen device, a processor located within the touch screen device may detect the user input and determine the substrate classification data and the positions of each spatial substrate based on the detected user input. The determined data may then be provided to a computing device that implements method 200. When the user input is detected via a mouse, keyboard, or trackball attached to the computing device that implements method 200, the classification data and positions of the substrates may be determined using a processor(s) housed within the computing device. Blocks 202 and 204 may be executed at any point before block 206 through 212, after block 212 or 214, or before block 216.

[0130] In block 206, data of the robot's working space may be provided to the routine implementing method 200 via a communication interface. The data of the robot's working space may be provided by scanning the robot's working space using a scanning device (or sensor system) attached to the movable robot member of the robot, as described in relation to FIG. 7 below. The data of the robot's working space may include not only color data such as RGB or still image data, but also 3D point cloud data. The computing device implementing method 200 may be used to control the robot constituting the scanning device via a robot controller, as described in relation to FIG. 7 below. For example, the computing system may allow a user to issue commands (plural) regarding the scan of the robot's workspace to the robot controller via the computing device. Also, the scan of the robot's working space may be automatically started, for example, at the start of the method of the present invention, by the computing device issuing respective commands to the robot controller. Thereafter, the robot controller can control the robot constituting the scanning device to scan the workspace, and the data acquired by the scanning device can be provided to the computing device via the robot controller.

[0131] In block 208, the routine implementing method 200 may determine the collision geometry existing in the working space based on the data of the robot's working space provided in block 206. The collision geometry may be generated by filtering the provided data of the robot's working space to reduce the number of data points, determining whether the data contains geometries of a specific size, creating 3D object(s) existing in the working space from extreme points, and filling the generated object(s) with volume to obtain the collision geometry.

[0132] In block 210, the routine implementing method 200 may determine scan path data for a scan path to be traced by a scanning device along the surface of each spatial substrate based on the determined position and determined collision geometry of each spatial substrate within the working space of the robot, and may provide the determined scan path data to the scanning device via a communication interface. If the spatial substrate is present in multiple zones of the spray booth, or if multiple spatial substrates are present in different zones, in order to ensure that the entire surface of each spatial substrate is scanned by the scanning device, the zones may be passed through in a predefined order during the generation of the scan path data. The determined scan path data may be provided via a communication interface to a scanning device, such as the sensor device 706 described in connection with FIG. 7, and the scanning device may scan the surface of each spatial substrate based on the provided scan path data. The scanning device may be the same scanning device used to acquire the data of the working space of the robot provided in block 206, or may be a different scanning device. The scanning device may be attached to a movable robotic member of the robot, and the determined scan path data may be provided from the computing device implementing method 200 to a robot control device that controls the robot including the scanning device. By performing the scan, detailed information regarding the spatial substrate is provided, and tool path data can be generated that results in a consistently high optical quality in terms of the appearance of the resulting coating.

[0133] In block 214, the routine implementing method 200 may obtain data indicating the shape and color of each spatial substrate acquired by the scanning device based on the scanning path data determined in block 212 and provided to the scanning device. For this purpose, the acquired scan path data may be assigned to a location ID, and the routine may associate the location ID with the determined location of each spatial substrate in order to assign the acquired scan path data to each spatial substrate. The acquired data may include 3D point cloud data and RGB color data or still image data. The acquired scan data may be stored by the scanning device or by a controller that controls the scanning device, such as a robot controller or a further computing device, and the data may be retrieved from the storage. The acquired scan data may be provided in real time to the computing device implementing method 200, and this computing device may store the received scan data.

[0134] In block 216, the routine implementing method 200 may generate spatial substrate data for each spatial substrate by combining the data retrieved in block 214 with the determined substrate classification data for each, and each data indicates the identity of the spatial substrate and each rule set for smoothing the surface of the generated 3D model. The rule set may be retrieved using the substrate classification data determined in block 204, for example, using a database containing the rule set, and each rule set is associated with the appropriate substrate classification data. The spatial substrate data for each spatial substrate may be generated by combining the data retrieved in block 214 with the determined substrate classification data for each. The spatial substrate data generated in block 214 for each spatial substrate may be stored in a data storage medium such as an internal memory before using the data in further blocks described in relation to FIG. 1. After the end of block 216, method 200 may proceed to block 104 of FIG. 1 described above.

[0135] Figures 3A and 3B show a flowchart of a method 300 for generating tool path data described in connection with block 114 of FIG. 1. The method 300 may be executed by the computing device described in connection with FIG. 1, and the computing device comprises at least one processor implementing a routine for performing the steps described in connection with the following blocks 302 to 326.

[0136] In block 302, the routine implementing the method 300 may generate a three-dimensional (3D) model for each spatial substrate based on the spatial substrate data provided in block 102 of FIG. 1. The provided spatial substrate data may be generated as described in connection with FIG. 2 above. The spatial substrate data may include, for each spatial substrate, 3D point cloud data, still image data of the spatial substrate, and a rule set for smoothing the surface of the generated 3D model. Generation of the 3D color model from the 3D point cloud data and the still image data may be performed using generally known methods and software such as the open-source library Open3D.

[0137] In block 304, the routine implementing method 300 may search each rule set and apply the searched rule set to smooth the surface of the 3D model generated in block 302 for each spatial substrate, and this step is generally optional. The rule set may be included in the spatial substrate data provided in block 102 of FIG. 1. The routine implementing method 300 may search for a rule set from a data storage medium including a rule set that is mutually related to the substrate classification data based on the substrate classification data included in the provided spatial substrate data. Smoothing of each generated 3D model may include determining the surface (or region) of the 3D model that matches the surface data included in the rules of the rule set, and applying an optimization algorithm such as a numerical smoothing algorithm known in the art in combination with the threshold value included in the rule to the determined surface. The threshold value may be used to control the degree of smoothing performed by the optimization algorithm. For example, when the type of the spatial substrate is a hatch, the corners are made smoother and the internal edges and cavities for taillights are determined, while for a front quarter panel, the corners are kept sharper and holes that may be treated as internal edges are ignored. By using a rule set to smooth the 3D model, the degree of smoothing can be controlled, and as a result, the quality of the coating can be controlled in terms of appearance. A high smoothness may be used for surfaces that do not require high quality, such as the front light pocket existing within the fender, and a low smoothness may be used for surfaces that require high quality, such as the part of the fender that is visible after attaching the coated fender to the vehicle body. The mesh of the 3D model obtained from the point cloud data is converted into a surface, which can be processed more efficiently, for example, when generating tool path data by a computer processor that generates tool path data as described later. For this reason, smoothing of the 3D model enables more effective generation of tool path data.

[0138] In block 306, the routine implementing method 300 may determine outer edge(s), edge(s) adjacent to open space, main surface, and open space existing within each spatial substrate based on the coating procedure data retrieved in block 108 of FIG. 1, and each 3D model generated in block 302 or each 3D model smoothed in block 304. For this purpose, the routine may search for an appropriate rule set and apply the rule set to each generated / smoothed 3D model. The routine may obtain an algorithm from the rule set and apply the algorithm to each smoothed 3D model to determine, as described above, outer edges, edge(s) adjacent to open space(s), main surface, and open space(s).

[0139] In block 308, the routine implementing method 300 may generate tool path data for outer edges and tool path data for edges adjacent to open space(s) based on the coating tool parameter data retrieved in blocks 106 and 108 of FIG. 1, the coating procedure data related to the first coating material (base coat material in this example), the substrate type data retrieved in block 110 of FIG. 1, and each 3D model generated in block 302 or each smoothed 3D model generated in block 304. The tool path data for outer edges and the tool path data for edges adjacent to open space(s) may be generated by: - determining the target distance(s), overlap ratio(s), pattern size, and rotation tolerance(s) of the coating tool based on the outer edges, edges adjacent to open space, and open space determined in block 306, the retrieved coating tool parameter data, the retrieved coating procedure data, the retrieved spatial substrate data, and each generated and smoothed 3D model; - generating tool path data from the determined target distance, overlap ratio, pattern size, and rotation tolerance of the coating tool using each generated and smoothed 3D model.

[0140] For example, when the spatial substrate is a fender or a bonnet, open spaces existing within the fender or bonnet, such as marker light pockets within the fender or hood scoop pockets within the bonnet, need not be processed according to the rules for coating the open spaces included in the substrate type data. The front light pockets, the upper inner rails and tabs of the dog leg, and the lower flange may be determined as open spaces to be painted according to the rules included in the painting procedure data. When the spatial substrate is a door, open spaces existing within the door excluding the window, such as door handle mounting holes, need not be processed according to the rules regarding the coating of the open spaces included in the substrate type data. The door jamb and window frame may be determined as open spaces to be painted according to the rules included in the painting procedure data.

[0141] In block 310, the routine implementing method 300 may generate tool path data for the main surfaces of each spatial substrate based on the coating procedure data associated with the first coating material (in this example, the base coat material) retrieved at block 108 of FIG. 1, particularly the rule set for coating the main surface, and each 3D model generated at block 302 or each smoothed 3D model generated at block 304. The tool path data for the main surface may be generated as follows: - determining whether the main surface of each spatial substrate includes at least two distinct surfaces based on the retrieved coating procedure data and each generated and smoothed 3D model, - determining whether the main surface of each spatial substrate includes a convex shape and creating a skin around the spatial substrate(s) using the convex hull method, - determining the target distance, overlap rate, pattern size, and rotational tolerance of the coating tool based on the determined separation surface, determined convex shape, retrieved coating tool parameter data, retrieved coating procedure data, and each generated and smoothed 3D model, and - Generating tool path data from the determined target distance, overlap rate, pattern size, and rotational tolerance of the coating tool using each generated and smoothed 3D model.

[0142] A rule set for coating a main surface may include rules for determining the start of the coating procedure, rules regarding the coating direction, rules regarding the rotation of the coating tool within the tool path, and rules regarding the separation of surfaces. The rule set for coating the main surface may include more rules or fewer rules. The rule for determining the start of the coating procedure may include, for example, determining the corner of the substrate and proceeding in a direction that enables maintaining the vertical direction of the coating material reservoir attached to the spray applicator. Rules regarding the coating direction may include, for example, coating the spatial substrate from top to bottom, or from bottom to top as in the case of a door, or from the front edge to the rear edge as in the case of a hood, or from side to side, and / or restricting the tool path(s) with curvature with respect to the edge of the substrate such that the tool path(s) do not follow the curvature or only follow the curvature up to a certain extent. Rules regarding the rotation of the coating tool within the tool path may include, for example, restricting the rotation so that the coating tool does not change its orientation during the coating of the surface (or target area) of the spatial substrate. Rules regarding the separation of surfaces may include determining the separated surface and coating the separated, i.e., adjacent, surfaces. In this example, the presence of a separated surface (or different target area) is determined by determining whether the spatial substrate includes a surface having a certain relative angle between the resulting two faces and the radius of curvature. For example, if the determined relative angle is from 25° to 90° and the radius of curvature is from 1 inch to 10 inches, surface separation is given. By ensuring that both the range of the relative angle and the radius of curvature are satisfied, it is guaranteed that a small body line with a small radius of curvature but little difference in the angle between the resulting two faces is determined to be a separate surface, thus avoiding unnecessary separation of the surfaces of the spatial substrate. Coating separate faces may include coating the adjacent faces in an adjacent-next pattern to maintain a wet film on both adjacent faces.As a result, the freshly coated surface is wet to accept overspray, and it is ensured that adjacent surfaces to be coated still have a wet overspray when they are coated.

[0143] The determination of whether the spatial substrate includes a convex outer skin may be performed by determining the surface curvature of each spatial substrate from side to side based on each generated or smoothed 3D model. Such convex surfaces may exist, for example, on bumper covers or bonnets. For the convex surface of the spatial substrate, the tool path data of the convex surface may be generated using an outer shell obtained by applying a convex hull algorithm known in the art.

[0144] The step of determining whether the main surface of each spatial substrate is composed of at least two distinct surfaces and / or convex surfaces may be executed in any order.

[0145] In block 312, the routine implementing method 300 may calculate the coating material application for each spatial substrate using the tool path data generated in blocks 308 and 310. The calculation of each coating material application using the generated tool path data may include a simulation of the coating material application using the generated tool path data. By this simulation, a coverage map of each generated or smoothed 3D model can be calculated. The calculated coverage map(s) may be provided to, for example, a display device via a communication interface, and the display device displays the received coverage map(s) within a graphical user interface (GUI). The coverage map may include different colors to indicate whether predefined parameters are met. The coverage map includes the calculated coating material application amount and the searched predefined parameters and can be visually compared.

[0146] In block 314, the routine implementing method 300 may determine whether each coating of each spatial substrate resulting from the calculated coating material application (amount) to each spatial substrate meets at least one predefined parameter. For this purpose, the routine may compare the calculated coating material application of each spatial substrate with each retrieved defined parameter of each spatial substrate and determine whether the calculated coating material application is inside or outside the retrieved defined parameter. The predefined parameter may be retrieved by the routine from a database using the provided material coating data, such as the ID of the coating material or the type of the coating material. The database may include the said parameter related to the coating material ID or the coating material type. The dry film thickness and / or the wet film thickness or the range thereof, and / or the surface area to be coated or the range thereof may be retrieved as the predefined parameter. The coverage map(s) calculated in block 312 may be used to determine whether a specific target area of the 3D model of each spatial substrate is coated at an appropriate ratio of the coating material such that the wet film thickness and / or the dry film thickness of the resulting coating layer meets the predefined range of the retrieved wet film thickness and / or dry film thickness. Also, the coverage map may be used to control the thickness of the transparent coating material and the thickness of the coating in the part of the spatial substrate that requires specific specifications regarding transparency, such as an ADAS (Advanced Driver Assistance System) or a radar sensor.

[0147] If at least one predetermined parameter is not satisfied, the routine returns to block 308 and may repeat blocks 308 and 310 using the determination result of block 314. After repeating blocks 308 and 310 to generate new tool path data, the newly generated tool path data may be checked using the steps described in blocks 312 and 314 above. This loop may be repeated until, in block 314, it is determined that the generated tool path data satisfies at least one, and in particular all, of the predefined parameters. By executing blocks 312 to 314, it is guaranteed that the coating obtained by applying the coating material using the robot path data generated from the tool path data satisfies predefined quality parameters such as wet and / or dry film thickness, surface area to be coated, etc. If the routine implementing method 300 determines in block 314 that at least one, and in particular all, of the searched predefined parameters are satisfied, it may proceed to block 316.

[0148] In block 316, the routine implementing method 300 may determine whether to apply a further coating material separate from the first coating material. This determination may be made based on the data included in the coating material data provided in block 102 of FIG. 1. For example, if the data includes data for at least two different coating materials, the routine may use the information to determine the number and types of coating materials to be applied. If the routine determines to apply a further coating material, it may proceed to block 318. Otherwise, it may proceed to block 116 of FIG. 1 described above.

[0149] In block 318, the routine implementing method 300 may generate tool path data as described in relation to blocks 308 and 310 based on the coating procedure data and coating tool parameter data related to the further coating material.

[0150] In block 320, the routine implementing method 300 may calculate the coating material application using the tool path data generated in block 318, as described in connection with block 312.

[0151] In block 322, the routine implementing method 300 may determine whether the coating material application calculated in block 320 meets at least one, and in particular all, of the predefined parameters, as described in connection with block 316. If at least one of the predefined parameters is not met, the routine implementing method 300 may return to block 318 and optimize the generated tool path data, as described in connection with block 314. If at least one of the predefined parameters is met, the routine may proceed to block 324.

[0152] In block 324, the routine implementing method 300 may determine whether at least one additional coating material is to be applied, as described in connection with block 316. If the routine determines that at least one additional coating material is to be applied, it may proceed to block 318 described above; otherwise, it may proceed to block 116 of FIG. 1.

[0153] FIG. 4 shows a flowchart of a method 400 for generating the robot path data described in block 114 of FIG. 1. Method 400 may be executed by the computing device described in connection with FIG. 1, and the computing device includes at least one processor that implements a routine for performing the steps described in connection with the following blocks 402-410.

[0154] In block 402, the routine implementing method 400 may determine whether to sort the generated tool path data. This determination may be made based on the data included in the retrieved coating procedure data or based on the programming of the routine. For example, the retrieved coating procedure data may include a rule that determines that the tool path data generated for outer edges and edges adjacent to open spaces is to be executed prior to the tool path data generated for open spaces and major surfaces. Executing the tool path for outer edges and edges adjacent to open spaces prior to the tool path for open spaces and major surfaces is beneficial because coating the edges causes overspray on the major surfaces, but this overspray can be covered later by coating the major surfaces so as to avoid an adverse effect on the final overall appearance. Further, this avoids applying too much coating material to the edges, which is undesirable because too much coating material on the edges tends to sag, flow, and form heavy edges, which adversely affects the final overall appearance of the coated substrate. If the routine determines in block 402 to sort the tool path data generated in block 112 of FIG. 1, it may proceed to block 404; otherwise, it may proceed to block 406, which will be described later.

[0155] In block 404, the routine implementing method 400 may sort the tool path data generated in block 112 of FIG. 1 according to its programming or according to the rules retrieved. The retrieved rules may be included in the coating procedure data or may be retrieved by the routine from a data storage medium such as a database based on the provided coating material data. The generated tool path data may be sorted according to the rules included in the retrieved coating procedure data and according to the rules retrieved based on the provided coating material data. For this purpose, the routine may retrieve appropriate rules from the retrieved coating procedure data, apply the rules to the generated tool path data, and sort the tool path data as appropriate. The generated tool path data may include metadata indicating whether the tool path data was generated for an outer edge, an edge adjacent to an open space(s), an open space(s), or a main surface, which is for facilitating sorting of the generated tool path data using this metadata. When one or more spatial substrates are coated, the routine may sort the generated tool path data such that all tool paths related to a specific coating material type such as a base coat material are executed before all tool paths related to a further coating material such as a clear coat material. However, it is also possible to sort (rearrange) the tool path data such that one spatial substrate is completely coated before coating a further spatial substrate. The tool paths related to a specific coating material such as a primer coating material may be executed before the tool paths related to the base coat material and the clear coat material for each specific spatial substrate, i.e., the tool paths for the base coat material and the clear coat material or for each spatial substrate may be grouped and executed in order such that after applying the primer coating material to all spatial substrates, the base coat material and the clear coat material are applied for each spatial substrate.

[0156] In block 406, the routine implementing method 400 may generate robot path data using the tool path data generated in block 112, as described in connection with block 114 of FIG. 1.

[0157] In block 408, the routine implementing method 400 may determine whether to optimize the generated robot path data. This determination may be made according to the programming of the routine. Optimization of the generated or sorted robot path data may be beneficial to enable smoothing to make the movement of the robot more efficient and consistent. If the routine determines in block 408 to optimize the generated robot path data, it may proceed to block 410. Otherwise, it may proceed to block 116 of FIG. 1 described above.

[0158] In block 410, the routine implementing method 400 may optimize the generated robot path data. Optimization of the generated robot path data may be performed using available open-source libraries such as Descartes (a ROS-Industrial project for executing path planning on a poorly defined Cartesian trajectory), and software frameworks such as trajopt (a software framework for generating robot trajectories by local optimization). After the end of block 410, the routine may proceed to block 116 of FIG. 1.

[0159] FIG. 5 shows a flow diagram of a second non-limiting embodiment of a method 500 for generating robot path data for a robot path (s) that a robot including a coating tool follows while coating at least a portion of the surface of a spatial substrate with at least one coating material. The method 500 may be used to coat at least a portion of the surface of a spatial substrate with a coating material using a robot system including a robot including a coating tool. The method 500 may include blocks 102 through 116 described in connection with FIG. 1, as well as additional blocks 502 through 506 described below. The method 500 may be executed by a computing device described in connection with FIG. 1, the computing device including at least one processor implementing a routine that executes the steps described in connection with the following blocks 502 through 506. The method 500 may be executed by a server device connected to a computing device that executes at least a portion of the blocks of FIG. 1 via a communication interface. In this setup, the computing device may function as a client device and may provide the generated tool path data and the provided coating material data to the server device. Next, the server device may perform calculations using the received data and provide the calculation results to the client device. This setup may be executed when the computing power of the computing device is not sufficient to perform the calculations.

[0160] In block 502, the routine implementing method 500 may determine whether to calculate the amount of each coating material required to coat at least a portion of the surface of each spatial substrate. This determination may be made according to the programming of the routine, or, for example, by displaying a graphical user interface that prompts the user to select whether to perform the calculation, or by providing respective buttons / menu items on the graphical user interface that trigger the calculation upon detection of user input indicating selection of the button / menu item. If the routine determines to calculate the amount of each coating material, it may proceed to block 504. Otherwise, method 500 may end, or it may proceed to block 102 of FIG. 1.

[0161] In block 504, the routine implementing method 500 may calculate the amount of each coating material required to coat at least a portion of the surface of each spatial substrate based on the tool path data generated in block 112 of FIG. 1 and the coating material data provided in block 102 of FIG. 1.

[0162] In block 506, the routine implementing method 500 may provide the calculated amount of each coating material for each spatial substrate. The calculated amount may be provided to a display device for display within a graphical user interface so that the user can prepare the required amount. The calculated amount may also be provided, along with the provided coating material data, via a communication interface to an automatic mixer that automatically prepares the calculated amount using the received data.

[0163] By executing blocks 504 and 506, the user can obtain information regarding the amount of coating material required for the painting process, and thus use this data for inventory planning or determine the amount of coating material that needs to be prepared for the painting procedure. In the latter case, since there is no need to prepare more paint than the amount required for the painting process, waste and costs associated with unnecessary paint can be reduced. Further, the determined amount and information regarding the coating material included in coating material data such as coating material ID can be provided to an automatic mixer, and the automatic mixer can mix the amount determined based on the received data. Thereby, the coating process can be fully automated, and waste of coating materials due to mixing errors can be prevented.

[0164] Blocks 502 to 506 may be executed after block 116 of FIG. 1. Blocks 502 to 506 may be executed after block 114 and before block 116 of FIG. 1. Thereby, the generated robot path data can be provided together with the calculated amount of each coating material.

[0165] FIG. 6 shows a computing device 600 that may be used to implement any of the methods described in the above figures. For example, a computing device 600 of the type shown in FIG. 6 may be used to implement the methods described in relation to FIGS. 1 to 5. In all cases, the computing device 600 represents a physical and specific processing mechanism.

[0166] The computing device 600 may include one or more hardware processors 602. The hardware processor(s) may include, but are not limited to, one or more central processing units (CPUs), and / or one or more graphics processing units (GPUs), and / or one or more application specific integrated circuits (ASICs), etc. More generally, any hardware processor may correspond to a general-purpose processing unit or a special-purpose processor unit.

[0167] Computing device 600 may also include a computer-readable storage medium 604 corresponding to one or more computer-readable media hardware units. The computer-readable storage medium 604 holds all kinds of information 606 such as computer or machine-readable instructions, settings, data, etc. The computer-executable instructions include, for example, instructions and data that, when executed by the processor 602, cause the computing device 600 to perform a specific function or group of functions. Alternatively or additionally, the computer-executable instructions may be configured to cause the computing device 600 to perform a specific function or group of functions. The computer-executable instructions may be, for example, binary, or intermediate format instructions such as assembly language, or instructions that undergo some translation (such as compilation) before direct execution by the processor, such as source code. By way of non-limiting example, the computer-readable storage medium 604 may include, for example, one or more solid-state devices, one or more magnetic hard disks, one or more optical disks, magnetic tapes, etc. Any instance of the computer-readable storage medium 604 may use any technology for storing and retrieving information. Further, any instance of the computer-readable storage medium 604 may represent a fixed or removable component of the computing device 600. Further, any instance of the computer-readable storage medium 604 may provide either volatile or non-volatile retention of information.

[0168] Computing device 600 may utilize any instance of computer-readable storage medium 604 in different ways. For example, any instance of computer-readable storage medium 604 may represent a hardware storage device (such as random access memory (RAM), etc.) for storing temporary information during the execution of a program by computing device 600, and / or a hardware storage device (such as a hard disk, etc.) for holding / archiving information on a more permanent basis. In the latter case, computing device 600 also includes one or more drive mechanisms 608 (such as a hard drive mechanism) for storing and retrieving information from an instance of computer-readable storage medium 604.

[0169] Computing device 600 may perform any of the above-described functions when hardware processor(s) 602 execute computer-readable instructions stored in any instance of computer-readable storage medium 604. For example, computing device 600 may execute computer-readable instructions to perform each block of the methods described in FIGS. 1-5.

[0170] Alternatively, or in addition, computing device 600 may rely on one or more other hardware logic components 610 to execute operations using a task-specific collection of logic gates. For example, the hardware logic component(s) 610 may include a fixed configuration of hardware logic gates that are created and set, for example, at manufacturing time and then cannot be changed. Alternatively, or in addition, the other hardware logic components 610 may include a set of programmable hardware logic gates that can be configured to execute different application-specific tasks. Devices in the latter category include, but are not limited to, programmable array logic devices (PALs), generic array logic devices (GALs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs), and the like.

[0171] FIG. 6 generally shows that the hardware logic circuitry 612 includes any combination of hardware processor(s) 602, computer-readable storage medium 604, and / or other hardware logic component(s) 610. That is, computing device 600 may employ any combination of hardware processor(s) 602 that execute machine-readable instructions provided on computer-readable storage medium 604 and / or one or more other hardware logic component(s) 610 that execute operations using a fixed and / or programmable collection of hardware logic gates. More generally, the hardware logic circuitry 612 corresponds to any type of one or more hardware logic components that execute operations based on logic stored in the hardware logic components and / or logic embodied in the hardware logic components.

[0172] In some cases (e.g., when computing device 600 represents a user computing device), computing device 600 also includes an input / output interface 614 for receiving various inputs (via input device 616) and providing various outputs (via output device 618). Exemplary input devices include a keyboard device, a mouse input device, a touch screen input device, a digitizing pad, one or more still image cameras, one or more video cameras, one or more depth camera systems, one or more microphones, a speech recognition mechanism, any motion detection mechanism (e.g., an accelerometer, a gyroscope, etc.), and the like. One particular output mechanism may include a display device 618 and an associated graphical user interface presentation (GUI) 620. The display device 618 may correspond to a liquid crystal display device, a light emitting diode display (LED) device, a cathode ray tube device, a projection mechanism, or the like. Other output devices include a printer, one or more speakers, a tactile output mechanism, an archive mechanism (for storing output information), and the like. Computing device 600 may also include one or more network interfaces 622 for exchanging data with other devices such as robot controller 704 and databases 708, 710, 712, and 722 of FIG. 7 via one or more communication conduits 624. One or more communication buses 628 communicatively couple the above-described components.

[0173] Communication conduit(s) 624 can be implemented in any way, for example, by a local area computer network, a wide area computer network (e.g., the Internet), a point-to-point connection, or any combination thereof. Communication conduit(s) 624 can include any combination of a hardwired link, a wireless link, a router, a gateway function, a name server, etc., governed by any protocol or combination of protocols.

[0174] FIG. 6 shows a computing device 600 composed of a discrete collection of separate units. In some cases, the collection of units may correspond to individual hardware units provided in the housing of a computing device having any form factor.

[0175] FIG. 7 shows a block diagram of a robot system 700 for coating at least a portion of the surface of a spatial substrate with at least one coating material according to an embodiment of the present invention. The robot system may be used to implement the method of the present invention, such as method 100 described in connection with FIG. 1, by generating robot path data using a computing device 718 and providing the generated robot path data to a robot device 724 via a robot controller 704.

[0176] The robot system 700 may include a computing device 710 and a robot device 702. The computing device 710 may be a computing device 600 as described in FIG. 6 and may include a CPU 712 and a storage medium 714. The CPU may be a hardware processor or other hardware logic component as described in connection with FIG. 6 above. The storage medium 714 may be any computer-readable storage medium as described in connection with FIG. 6 above. The computer-readable storage medium may store computer-readable instructions, such as program code, that cause the computing device 710, particularly the CPU 712, to execute the methods of the present invention, such as method 100 described in connection with FIG. 1 and method 500 described in connection with FIG. 5.

[0177] Computing device 710 may be connected to display device 726 via a communication interface. Display device 726 may receive data such as a 3D model generated by block 302 or 304 described in relation to FIG. 3A, generated tool path data, a coverage map calculated by block 312 of FIG. 3A and / or block 324 of FIG. 3B, and / or the amount of coating material calculated by block 504 of FIG. 5 from the computing device, and may display the received data through a graphical user interface. For this purpose, either display device 726 or computing device 710 may generate a user interface presentation including the respective data, and may provide / display the generated user interface presentation. Display device 726 may be a mobile display device as shown in FIG. 7, or may be a stationary display device such as a computer monitor. The display device may include a screen such as an LCD screen to display a graphical user interface including data generated by computing device 710.

[0178] Computing device 710 may be connected to different databases 716, 718, 720, 722, and 724 via a communication interface. Database 716 may include spatial substrate data, database 718 may include coating material data, database 720 may include coating parameter data, database 722 may include coating procedure data, and database 724 may include substrate type data as described above. The data included in the databases may be mutually associated with data that enables computing device 710 to search for the data. Some of the data stored in databases such as spatial substrate data may be stored in storage medium 714 instead of the database. At least some of the data may be stored in the same database to reduce the number of databases connected to the computing device.

[0179] The computing device 710 may be connected to a further computing device (not shown) different from the computing device 710, such as a server device. The server device may be used, for example, to determine the presence of masking material on the spatial substrate, or to execute further blocks of the method of the present invention, such as the method 100 described in connection with FIG. 1.

[0180] The system may further include a robot device 702 including a robot controller 704 and a robot arm (or movable robot member) 706. The robot device 702 may be connected to the computing device 710 via the robot controller 704 using a communication interface. The robot control device 704 may be configured to receive robot path data and optionally further commands (such as commands directed to tool changes), and to control, i.e., move, the robot using the received robot path data and further commands. The robot controller may be disposed outside the spray booth (see FIG. 9) to avoid adverse effects on the robot controller during the spraying operation. The robot arm 706 may be any industrial robot having "n" degrees of freedom to which a sensor system or a coating tool 708 can be automatically attached. The robot arm 706 may be disposed within the spray booth (see FIG. 9). The robot arm may include either a sensor system or a coating tool 708. The robot arm 706 may not include a sensor system or a coating tool 708. In this case, the system or the coating tool 708 may be stored in a tool rack (see FIG. 12B). The robot arm 706 may include a tool changer configured to attach different coating tools or sensor systems to the robot arm 706. A suitable coating tool or sensor system 708 may be picked up by the robot arm 706 by providing the position of the tool in the tool rack to the robot controller 704.

[0181] The sensor system 708 may be configured to generate data of the robot's working space and acquire data regarding the shape and color of the spatial substrate based on the scan path data provided to the robot controller 704. The scan path data may be determined, for example, as described in relation to block 210 of FIG. 2. The sensor system may include a depth sensor such as a laser scanner.

[0182] The coating tool 708 may include a coating applicator such as the applicator shown in FIGS. 8 and 8B described below. The coating tool may further include a coating material reservoir containing a specific coating material as shown in FIGS. 12A and 12B.

[0183] FIG. 8A shows the z-axis of the spray applicator 802 of the coating tool according to an embodiment of the present invention. The spray applicator 802 may be an electrostatic spray applicator including a nozzle similar to the rotary bell 804. As shown in FIG. 8A, the target distance may be the distance between the end of the nozzle of the spray applicator 802 and the surface of the spatial substrate 806. FIG. 8B illustrates the X-axis and Y-axis of the spray applicator 802 of the coating tool of FIG. 8A. FIG. 8B shows a top view of the spray applicator 802 of FIG. 8A and illustrates the movement of the spray applicator 802 in the vertical and horizontal directions.

[0184] FIG. 9 is a schematic diagram of an exemplary system 900 including a spray booth 906 and a robot system of the present invention such as the robot system 700 of FIG. 7.

[0185] The system 900 may include an electrical cabinet 902 that houses the power required for the use of the system 900. The system 900 may further include a cooler 904, similar to the heat exchanger 916 for purge air for robotic devices such as the robotic arm 706 and / or the coating tool 708 described in relation to FIG. 7.

[0186] System 900 may further include a spray booth 906 including a spray booth door 908 that enables placement of a spatial substrate within the spray booth and removal of the coated spatial substrate from the spray booth. The spray booth may include heating means (not shown) such that coating material(s) applied by a robotic arm can be dried and / or cured within the spray booth without removing the coated spatial substrate or placing the coated spatial substrate in another oven.

[0187] System 900 may further include a spray booth ventilation 910 that enables ventilation of the spray booth to remove residues of coating materials or volatile compounds that evaporate from the applied coating material during drying and / or curing of the resulting coating film.

[0188] System 900 may further include a robot controller 912 external to the spray booth 906. Thereby, it is ensured that the robot controller 912 is not adversely affected by spray mist, similar to the evaporating volatile compounds present within the spray booth during spraying and curing. The robot controller 912 may be connected to a robot 914. The robot 914 may include a robotic arm (see FIG. 11). Suitable robots include robots configured for spraying operations, such as the FANUC P-50iB / 15, a 6-axis industrial robot commercially available from Fanuc. The robot may be fixed to a Güdel rail system to enable movement of the robot within the spray booth.

[0189] Figure 10 shows a top view of the spray booth 906 and the robot controller 912 of FIG. 9. The robot 914 may be fixed on a bar 1002 that enables the movement of the robot along the length of the spray booth 906 using a Gudel rail system so that the robot can reach possible positions of the spatial substrate within the spray booth 906. The spray booth may further include an air outlet 1004 that allows air supplied into the spray booth from the ceiling to flow out through the air outlet. A gantry axis motor configured to move the robot along the spray booth may be present within an explosion-proof box 1006 to avoid an explosion during the spraying of a solvent-based liquid coating material containing explosive liquids.

[0190] Figure 11 is a partial side view of the spray booth 906 of FIG. 9. A sensor system 706 may be attached to the robot 914. The robot 914 may not have a sensor system or a coating tool attached thereto. The robot 914 may have a coating tool (see FIGS. 12A and 12B) attached thereto. The Gudel rail system 1102 may be used to move the robot 914 as described above.

[0191] Figure 12A shows a partial side view of the spray booth 906 of FIG. 9, including a tool rack that houses a plurality of coating tools, a housing that houses a sensor system, and the Gudel rail system 1102.

[0192] The tool rack 1202 may include a plurality of coating tools 706. Each coating tool may include a spray applicator 802 and a coating material reservoir 1206 that contains a specific coating material (see FIG. 12B). Each coating tool may be automatically attached to the robot 914 (not shown) using a tool changer.

[0193] The sensor device 706 may be housed within the housing 1204 to avoid contamination of the sensor device during a spraying procedure that may contaminate the sensor device 708. The robot 914 may be configured to open the housing and remove the sensor device from the housing 1204 using a tool changer.

[0194] FIG. 12B is an enlarged view of a tool rack and a housing that houses the sensor system shown in FIG. 12A. The tool rack may include a plurality of coating tools such as four coating tools. Each coating tool may include a spray applicator 802 described in connection with FIG. 8 above and a coating material reservoir 1206. The coating material reservoir may contain a specific coating material that may be prepared, for example, by mixing different materials such as pigment paste, base varnish, thinner, hardener, etc. within the reservoir or by mixing the materials within a mixing device and filling the resulting coating material into the reservoir. Each coating tool may be picked up by the robot 914 based on instructions provided by the robot controller 912.

[0195] The present disclosure has been described in conjunction with preferred embodiments by way of example. However, other variations can be understood and implemented by those skilled in the art, who practice the claimed invention, from a study of the drawings, the present disclosure, and the claims. It should be noted that it is not essential that different steps be performed at a particular location or at one node of a distributed system, i.e., each step may be performed at different nodes using different devices / data processing units.

[0196] As used herein, "determine" includes "initiate or cause to determine", "generate", "query", "access", "correlate", "match", and "select" includes "initiate or cause to initiate generating, accessing, querying, correlating, selecting and / or matching", and "provide" includes "initiate or cause to initiate determining, generating, accessing, querying, correlating, selecting and / or matching, transmitting and / or receiving". "Initiate an action, or cause to be executed" includes any processing signal that triggers the computing processor to execute each respective action.

[0197] In the claims and specification, the terms "comprising" or "including" do not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. A single element or other unit may perform the functions of a plurality of entities or items recited in the claims. The mere fact that certain means are recited in mutually different dependent claims does not indicate that combinations of these means cannot be used in advantageous embodiments.

Claims

1. A computer-implemented method for generating robot path data for a robot path followed by a robot including a coating tool while coating at least a part of the surface of at least one spatial substrate with at least one coating material, the following steps: (a) via a communication interface, to at least one computer processor, - spatial substrate data including substrate classification data and data indicating the shape and color of each spatial substrate, - coating material data including data indicating the type of the at least one coating material, and optionally data indicating the order of the coating materials applied to the spatial substrate, and / or data indicating the coating tool; providing step, (b) optionally, based on the provided spatial substrate data, determining by the at least one computer processor whether each spatial substrate includes at least one masking material; (c) using the at least one computer processor, via the communication interface, - coating tool parameter data based on the provided coating material data, the coating tool parameter data including coating tool tolerance data and at least one application parameter related to the coating tool, - coating procedure data including a rule set for coating outer edges and edges adjacent to open spaces and a rule set for coating main surfaces, - substrate type data based on the provided spatial substrate data, the substrate type data including a rule set regarding the type of spatial substrate that matches the substrate classification data; searching step, (d) generating, by the at least one computer processor, tool path data for a tool path followed by the coating tool along the surface of each spatial substrate based on the data obtained in step (c) and optionally the result of the determination performed in step (b); (e) generating, by the at least one computer processor, robot path data based on the tool path data generated in step (d), Step (f) of providing the generated robot path data via the communication interface A computer-implemented method comprising the above. **Claim 2** The method according to claim 1, wherein the data indicating the shape and color of each spatial substrate includes data representing each spatial substrate in three-dimensional space, particularly the three-dimensional point cloud of each spatial substrate, and the color data of each spatial substrate. **Claim 3** The step of providing spatial substrate data includes the following - A step of detecting, by the at least one computer processor, a user input indicating a substrate classification associated with each spatial substrate and a user input indicating the position of each spatial substrate in the working space of the robot - A step of determining, by the at least one computer processor, based on the detected user input, substrate classification data for each spatial substrate and the position of each spatial substrate in the working space of the robot - A step of providing, via a communication interface, data of the working space of the robot to the at least one computer processor - A step of determining, by the at least one computer processor, the collision geometry existing in the working space based on the provided data of the working space of the robot - A step of determining, by the at least one computer processor, based on the determined position of each spatial substrate in the robot's workspace and the determined collision geometry, scan path data for a scan path to be traced by a scanning device along the surface of each spatial substrate, and providing the determined scan path data to the scanning device via the communication interface - A step of generating, by the at least one computer processor, spatial substrate data for each spatial substrate by searching for data indicating the shape and color of the spatial substrate obtained by the scanning device based on the provided scan path data via the communication interface and combining the retrieved data with at least the determined substrate classification data for each substrate The method according to claim 1 or 2, comprising the above. **Claim 4** The method according to claim 1 or 2, wherein the data indicating the identity of the at least one coating material includes the name of each coating material type, the ID of each coating material type, or a combination thereof. **Claim 5** The method according to claim 1 or 2, wherein the tolerance data of the coating tool includes target distance data, overlap rate data, pattern size data, rotational tolerance regarding the z-axis of the coating tool, rotational tolerance regarding the x-axis of the coating tool, rotational tolerance regarding the y-axis of the coating tool, or a combination thereof.

6. The method according to claim 1 or 2, wherein the rule set for coating an outer edge and an edge adjacent to an open space comprises at least one algorithm for determining an outer edge and at least one algorithm for determining an edge adjacent to an open space existing within the surface of a spatial substrate.

7. The method according to claim 1 or 2, wherein the rule set for coating a main surface includes rules for determining the start of a coating procedure, rules regarding a coating direction, rules regarding the rotation of a coating tool within a tool path, rules regarding the separation of surfaces, or a combination thereof.

8. The method according to claim 1 or 2, wherein at least one rule set for a type of spatial substrate that matches the substrate classification data included in provided spatial substrate data includes at least one rule for coating an edge adjacent to an open space for each type of spatial substrate, data regarding the required quality of a tool path, optionally at least one rule for coating an open space within a spatial substrate, and optionally at least one rotational tolerance of a coating tool.

9. The step of generating tool path data for a tool path traced by a coating tool along the surface of each spatial substrate comprises the following steps: - Generating, by the at least one computer processor, a 3D model of each spatial substrate based on the provided spatial substrate data and, optionally, applying a rule set for smoothing the surface of each generated 3D model; - Determining, by the computer processor, an outer edge, an edge adjacent to an open space, a main surface, and an open space existing within each spatial substrate based on the retrieved coating procedure parameter data and the generated and optionally smoothed 3D models. - By the at least one computer processor, based on the retrieved coating procedure data, the retrieved coating tool parameter data, the retrieved substrate type data, the determined outer edges, the edges adjacent to the open spaces, and the open spaces, and the generated and optionally smoothed 3D model, generating tool path data for the outer edges and tool path data for the edges adjacent to the open spaces; - By the at least one computer processor, based on the retrieved coating procedure data, the retrieved coating tool parameter data, the determined main surfaces, and the generated and optionally smoothed 3D model, generating tool path data for the main surfaces of each spatial substrate; - Optionally, repeating the steps for at least one additional coating material based on the retrieved coating procedure data and the retrieved coating tool parameter data; The method according to claim 1 or 2, comprising.

10. Generating robot path data includes determining collision geometry within the working space of the robot based on the spatial substrate data, and determining robot path data based on the determined collision geometry and the generated tool path data, according to the method of claim 1 or 2.

11. Generating robot path data is the following step - Using the at least one computer processor and, prior to generating the robot path data, sorting the generated tool path data such that the robot path generated from the tool path data for the outer edges and the edges adjacent to the open spaces is executed before or after the robot path generated from the tool path data for the main surfaces, and / or - Optimizing the generated or sorted robot path data by the computer processor; The method according to claim 1 or 2, further comprising.

12. A computing device for generating robot path data for a robot path followed by a robot including a coating tool when coating a spatial substrate with at least one coating material, comprising: - At least one computer processor, and - A memory that stores instructions that, when executed by a processor, configure the apparatus to perform the steps of claim 1. A computing apparatus including the same. **Claim 13** A robotic system for coating at least one surface of a space substrate with at least one coating material, comprising: - The computing apparatus of claim 12 for generating robot path data for a robot path that a robot of the robotic system follows while coating at least one surface of the space substrate with at least one coating material. - A robot apparatus configured to receive the generated robot path data and use the received robot path data to apply at least one coating material from a coating tool to at least a portion of a surface of the space substrate. A system including the same. **Claim 14** Use of the method of claim 1 or the computing apparatus of claim 12 for coating at least a portion of a surface of a space substrate with a coating material using a robotic system including a robot including a coating tool. **Claim 15** A non-transitory computer-readable storage medium including instructions that, when executed by a computer, cause the computer to perform the steps of the method of claim 1 or 2.

Images