Method and device for generating robotic path data for automatically coating at least portion of surface of spatial substrate with at least one coating material

Abstract

Aspects described herein generally relate to a method for generating robot path data for robot path (s) that a robot comprising a coating tool is to follow during coating at least a portion of a surface of a spatial substrate with at least one coating material spatial substrate, and a corresponding apparatus, or computer element. Further, aspects described herein generally relate to a robotic system for coating at least one surface of a spatial substrate with at least one coating material. The method, the corresponding device, or the computer element according to the invention allows the automatic application of the coating material to a substrate having a large geometry change and provides consistency of the application compared to, for example, manual application of the coating material during a repair process of an automobile or an automobile part, and the manual application depends on the painter executing the application to a great extent.

Description

Technical Field

[0001] The various aspects described herein generally relate to a method, and corresponding apparatus, or computer element, for generating robot path data for (multiple) robot paths to be followed by a robot including a coating tool during coating at least a portion of the surface of a space substrate with at least one coating material. In addition, the various aspects described herein generally relate to a robot system for coating at least one surface of a space substrate with at least one coating material. More specifically, the various aspects described herein relate to a method, and corresponding apparatus, or computer element for generating robot path data using color and geometry data of a space substrate and different rule sets including rules for coating outer edges and edges adjacent to (multiple) open spaces and rules for coating main surfaces. The use of 3D data (which can be acquired by the robot using a scanning device attached to the robot) allows the geometry and color of the space substrate to be coated to be determined before the coating process, and thereby allows the generation of robot path data for substrates that exhibit large geometric changes without the need for the data to exist before performing the method of the present invention. In addition, the various aspects described herein relate to a robot system comprising a computing device for generating robot path data according to the method of the present invention and a robot device for receiving the robot path data and coating the space substrate according to the received robot path data. The method, corresponding apparatus, or computer element of the present invention allows for the automated application of coating materials to substrates having widely varying geometries and provides for greater consistency of application compared to manual application of coating materials, for example during the repair process of an automobile or automobile part, which is largely dependent on the painter performing the application. Background Art

[0002] Vehicles, particularly land vehicles such as automobiles, motorcycles and truck bodies, are often treated with multiple layers of coatings that enhance the appearance of the vehicle and also provide protection against corrosion, scratches, cracks, UV rays, acid rain and other environmental conditions. Over the past two decades, cars and trucks have typically been treated with a multi-layer paint system comprising a basecoat layer(s) and a clearcoat layer.

[0003] The production of these multi-layer paint systems generally involves electrophoretic deposition of electrocoating materials on metal substrates (such as automobile bodies), and curing the applied electrocoating materials. The metal substrate can be subjected to various pretreatments before depositing the electrocoating materials, for example, by applying known conversion coatings, such as phosphate coatings, more specifically zinc phosphate coatings. Afterwards, fillers or primer-surfacer materials can be applied to the cured electrocoating and cured. If such a layer is present, at least one paint material containing color and / or effect pigments is applied to the cured layer. However, at least one paint material can also be applied directly to the cured electrocoating. For plastic substrates, primer materials can be applied before applying the paint materials to increase the adhesion of the multi-layer coating to the substrate. At least one paint film or the topmost paint film produced thereby is then coated with a clearcoat material without curing alone. The clearcoat film and all (multiple) paint films present are then cured together (so-called 2-coat-1-bake (2C1B) or 3-coat-1-bake (3C1B) methods).

[0004] When film defects (such as peeling, discoloration, scratches, etc.) occur in such a multi-layer paint system, they are usually repaired to restore the original appearance of the vehicle. If such film defects occur directly after OEM finishing, they are repaired directly at the OEM manufacturing site by so-called "OEM car repair". If such defects occur at a later point in time, they are usually repaired by so-called "car repair" in a car repair shop. The repair process can be roughly divided into edge-to-edge repair, blend-in process and spot repair. Edge-to-edge repair can be performed when the portion to be repaired of the multi-layer coating is relatively large, and usually involves removing the damaged portion of the multi-layer coating and repairing the entire area. Spot repair is performed when the portion to be repaired of the multi-layer coating is small or when the position of the portion to be repaired of the multi-layer coating is not in a prominent position.

[0005] Repairing the entire area usually includes cleaning and sanding, and if necessary, filling the damaged area. Then, if necessary, after further pretreatment, the damaged area and adjacent areas are usually coated with an opaque coating agent (such as a suitable paint material). After drying the resulting coating, the coating and adjacent areas are usually coated with a varnish composition, and then the varnish composition is dried together with the previously applied (multiple) coatings. Typically, point repairs involve sanding the point to be repaired, coating the surface with an opaque coating material, drying the applied coating material, sanding the applied coating material and applying a varnish material. In order to reduce the color mismatch of the repair area or point, the repair is "blended" outside the area or point itself. This is a process of reducing the thickness of the applied coating film while moving away from the repair area or point. Therefore, the color gradually changes from the (incorrect) color on the area or point to the (correct) color of the rest of the area. If this change is gradual enough, human vision will not detect the mismatch.

[0006] When not only the multi-layer coating but also the underlying substrate is damaged (e.g., during an automobile accident), the damaged part must be removed and a new part with the desired color must be attached during the automobile repair. This requires coating the entire part of the automobile with the multi-layer coating and then mounting the part(s) on the automobile.

[0007] The demands placed on vehicle refinishing today are extremely high. Therefore, the finished result must be comparable to the original paintwork after baking in terms of both visual and technical aspects, i.e. the repaired color must match the color of the rest of the vehicle so that the observer cannot distinguish the repaired areas. In addition, the mechanical properties of the repaired multilayer coating should be comparable to those of the original paintwork.

[0008] However, the resulting optical appearance of the applied coating material(s) depends to a large extent on the parameters used during the application of the coating material(s), i.e. different application conditions lead to different optical appearances of the coated automobile or automobile part. In order to reduce the influence of the application conditions on the resulting optical appearance of the coated substrate, a robotic system can be used that automatically applies the coating material(s) to the substrate. With a robotic painting system, a controller provides instructions (also referred to as robot path data) to the robot, so that the robot coats the corresponding substrate according to the provided robot path data.

[0009] When generating robot movements of complex shapes for spatial substrates, such as cars or parts thereof, the coating application should be as uniform and complete as possible. The generation of the complex robot movements can be performed automatically based on electronic data of the substrate, such as CAD data, and predefined coating rules for each type of substrate, such as car hoods, car fenders, car doors, etc. However, the method cannot be used for substrates for which CAD data are not available, such as substrates used in the field of car repair, whose geometric structures vary greatly.

[0010]

[0006] It would therefore be desirable to provide methods and systems that allow for automatic (ie, without human interaction) generation of robotic path data for spatially varying substrates to ensure consistency in coating material application and resulting optical coating quality. Summary of the invention

[0011] From a certain perspective, in order to solve the above problems, the following solutions are proposed:

[0012] A computer-implemented method for generating robot path data for a robot including a coating tool to follow during coating of at least a portion of a surface of at least one spatial substrate with at least one coating material, the method comprising the following steps:

[0013] (a) providing to at least one computer processor via a communication interface

[0014] - spatial substrate data for each spatial substrate, the spatial substrate data comprising substrate classification data and data indicative of the geometry and color of each spatial substrate, and

[0015] - coating material data, the coating material data comprising data indicating the type of the at least one coating material and optionally indicating the sequence of coating materials to be applied to each spatial substrate and / or data indicating the coating tool;

[0016] (b) optionally determining, using the at least one computer processor, whether each spatial substrate includes at least one shielding material based on the provided spatial substrate data;

[0017] (c) using the at least one computer processor to obtain via the communication interface

[0018] - coating tool parameter data based on the provided coating material data, said coating tool parameter data comprising coating tool tolerance data and at least one application parameter associated with the at least one coating material,

[0019] - coating program data comprising a set of rules for coating outer edges and edges adjacent to the open space(s) and a set of rules for coating the main surface, and

[0020] - substrate type data based on the provided spatial substrate data, the substrate type data comprising a set of rules for spatial substrate types to match the substrate classification data;

[0021] (d) generating, using the at least one computer processor, tool path data for the tool path(s) to be followed by the coating tool along the surface of each spatial substrate based on the data obtained in step (c) and optionally the results of the determination performed in step (b);

[0022] (e) generating, using the at least one computer processor, the robot path data based on the tool path data generated in step (d); and

[0023] (f) providing the generated robot path data via the communication interface.

[0024] Further disclosures:

[0025] A method for coating at least a portion of a surface of a space substrate with a coating material using a robotic system including a robot comprising a coating tool, the method comprising the steps of:

[0026] (a) providing to at least one computer processor via a communication interface

[0027] - spatial substrate data of the spatial substrate, the spatial substrate data comprising substrate classification data and data indicative of the geometry and color of the spatial substrate, and

[0028] - coating material data, the coating material data comprising data indicating the type of the at least one coating material and optionally indicating the sequence of coating materials to be applied to the spatial substrate and / or data indicating the coating tool;

[0029] (b) optionally determining, using the at least one computer processor, whether the space substrate includes at least one obscuring material based on the provided space substrate data;

[0030] (c) using the at least one computer processor to obtain via the communication interface

[0031] - coating tool parameter data based on the provided coating material data, said coating tool parameter data comprising coating tool tolerance data and at least one application parameter associated with the coating tool,

[0032] - coating program data comprising a set of rules for coating the outer edges and the edges adjacent to at least one open space and a set of rules for coating the main surface, and

[0033] - substrate type data based on the provided spatial substrate data, the substrate type data comprising a set of rules for spatial substrate types to match the substrate classification data;

[0034] (d) generating, using the at least one computer processor, tool path data for at least one tool path to be followed by the coating tool along the surface of the spatial substrate based on the data obtained in step (c) and optionally the result of the determination performed in step (b), wherein generating the tool path data for at least one tool path to be followed by the coating tool along the surface of each spatial substrate comprises:

[0035] - generating, with the at least one computer processor, a 3D model of the space substrate based on the provided space substrate data, and optionally applying a set of rules to smooth the surface of the generated 3D model,

[0036] - using the computer processor to determine one or more outer edges, one or more edges adjacent to an open space, a major surface and one or more open spaces present within the spatial substrate based on the acquired coating process parameter data and the generated and optionally smoothed 3D model,

[0037] - generating, with the at least one computer processor, tool path data for outer edges and tool path data for edges adjacent to at least one open space based on the obtained coating procedure data, the obtained coating tool parameter data, the obtained substrate type data, the determined outer edges, edges adjacent to open spaces and open spaces and the generated and optionally smoothed 3D model,

[0038] - generating, with the at least one computer processor, tool path data for the major surface of each spatial substrate based on the acquired coating procedure data, the acquired coating tool parameter data, the determined major surface and the generated and optionally smoothed 3D model, and

[0039] - optionally, repeating said steps for at least one further coating material based on the acquired coating process data and the acquired coating tool parameter data associated with the at least one further coating material;

[0040] (e) generating, using the at least one computer processor, the robot path data based on the tool path data generated in step (d); and

[0041] (f) providing the generated robot path data to a robot comprising the coating tool via the communication interface for coating at least a portion of the surface of the spatial substrate with the coating material.

[0042] The basic advantage of the method according to the invention is that, due to the use of a set of rules for coating outer edges and edges adjacent to (multiple) open spaces, a set of rules for coating the main surface, and a set of rules for each type of space substrate, the method according to the invention allows the generation of robot path data for space substrates with large geometric variations. These rule sets allow different coating rules to be applied to the identified features of the space substrate (such as outer edges, edges adjacent to (multiple) open spaces, and (multiple) open spaces) and / or the characteristics of the space substrate. This allows the use of a robotic coating system to coat space substrates with large geometric variations, such as automotive parts, in the field of automotive repair. Information about the color of the space substrate allows the automatic identification of specific areas of the space substrate, which are either excluded from the coating process (e.g., masking materials) or included (e.g., areas that have been coated with primer or primer-surfacer coatings). In addition, color information allows the identification of the boundaries of those colored areas, so that specific rules can be applied to achieve a smooth coating transition to improve the color and surface quality of the coated space substrate. The method of the present invention allows the generation of robot path data for one or more space substrates in one step, regardless of the coating material to be applied to the space substrate and the type of space substrate, making the method of the present invention very effective in fully automatically coating multiple space substrates with (multiple) coating materials. Due to the automation of the coating material application, a constant high optical and mechanical quality is achieved, while manual application would produce differences due to the influence of different application parameters on the resulting overall quality.

[0043] Further disclosures:

[0044] A computing device for generating robot path data for a robot including a coating tool to be followed during coating of a spatial substrate with at least one coating material, the computing device comprising:

[0045] - at least one computer processor; and

[0046] - a memory storing instructions which, when executed by the processor, configure the apparatus to perform the steps of the method of the invention.

[0047] Further disclosures:

[0048] A robotic system for coating at least one surface of a space substrate with at least one coating material, the system comprising:

[0049] a computing device according to the invention for generating robot path data of a robot path(s) to be followed by a robot of the robot system during coating of at least one surface of at least one spatial substrate with at least one coating material,

[0050] - A robotic device configured to receive the generated robot path data and to use the received robot path data to apply at least one coating material from a coating tool to at least a portion of the surface of the at least one spatial substrate.

[0051] Further disclosures:

[0052] A robotic system for coating at least one surface of a space substrate with at least one coating material, the system comprising:

[0053] - a computing device for generating robot path data of a robot path(s) to be followed by a robot of the robot system during coating of the at least one surface of the spatial substrate with at least one coating material, the computing device comprising:

[0054] at least one computer processor, and

[0055] a memory storing instructions which, when executed by the processor, configure the apparatus to perform the methods as disclosed herein,

[0056] - a robotic device configured to receive the generated robotic path data and to apply at least one coating material from a coating tool to at least a portion of the surface of the spatial substrate using the received robotic path data.

[0057] A basic advantage of the robot system according to the invention is that spatial substrates with large geometric variations can be coated with at least one coating material fully automatically without any user interaction during the generation of robot path data. By using a set of rules during the generation of robot path data, such robot path data can be reliably generated for substrates with large geometric variations, thus allowing such substrates to be coated by the robot system using the generated robot path data in order to obtain a coating with high optical and mechanical quality, regardless of the geometry of the respective substrate. If the robot system further comprises a scanning device, data on the color and geometry of the spatial substrate can be determined by the robot device, thereby reducing the number of separate devices required to determine the geometry and color of the spatial substrate (required for the computing device of the invention to determine the robot path data). In addition, the presence of the scanning device allows the acquisition of data on the color and geometry of the spatial substrate to be coated before the coating process, thereby allowing the coating of spatial substrates for which no data on its color and geometry are available before the coating process is performed. Information on the color of the spatial substrate can be used to automatically identify specific areas of the spatial substrate, which are either excluded from the coating process (e.g., masking materials) or included (e.g., areas that have been coated with a primer or primer-surfacer coating). Furthermore, the color information can be used to identify the boundaries of those colored areas so that specific rules can be applied to achieve smooth coating transitions to improve the color and surface quality of the coated space substrate. The robotic system of the present invention allows for the application of at least one coating material to one or more space substrates regardless of the type of coating material and space substrate to be applied to the space substrate, making the robotic system very effective in fully automatically coating multiple space substrates with (multiple) coating materials.

[0058] Further disclosures:

[0059] Use of the method according to the invention or the computing device according to the invention for coating at least a portion of the surface of at least one space substrate with at least one coating material using a robot comprising a coating tool.

[0060] Further disclosures:

[0061] A non-transitory computer-readable storage medium comprises instructions which, when executed by a computer, cause the computer to perform the steps of the method according to the present invention.

[0062] Any disclosures and embodiments described herein are related to the methods, systems, devices, and computer elements disclosed herein, and vice versa. The benefits provided by any embodiment and example provided herein are also applicable to all other embodiments and examples, and vice versa.

[0063] Embodiments of the method of the present invention

[0064] The method of the present invention allows the generation of robot path data of (multiple) robot paths to be followed by a robot including a coating tool (such as a spraying application tool) during coating at least a portion of the surface of at least one space substrate with at least one coating material. The generated robot path data can be used by a robot system, which includes a robot that includes a coating tool to coat at least a portion of the surface of a substrate. In one example, robot path data is generated for one space substrate. In another example, robot path data is generated for at least two space substrates (such as 2 to 5 or 2 to 10 space substrates). The term "robot path" as used herein refers to the movement path of a robot (such as an industrial robot) relative to the surface of a space substrate, in particular relative to the surface of the space substrate to be coated with a coating material applied from a coating tool attached to the robot. The term "coating material" as used herein refers to a chemical composition in liquid, paste or powder form, which, when applied to the surface of a space substrate, produces a coating having protective, decorative and / or other specific properties (see DIN EN 971-1: 1996-09). The coating produced may include one coating or multiple coatings (also referred to as a multilayer paint system). The coating (s) in the coating can be produced using different or similar coating materials. For example, the coating can include two paint layers, which can be prepared by applying two different paint materials or by applying the same paint material twice.

[0065] The robot can be any multi-axis industrial robot suitable for applying at least one coating material to the surface of a space substrate. The robot can include a movable robot member, such as a robot arm, and the coating tool can be attached to the movable robot member. The robot can be controlled using a robot controller, as described later.

[0066] The space substrate can have any shape. In one aspect, the space substrate is a vehicle part. The term "vehicle part" is to be understood in the present case in a broad sense and relates to a car (such as a car, a van, a minivan, a bus, an SUV (Sports Utility Vehicle)); a truck; a semi-trailer truck; a tractor; a motorcycle; a trailer; an ATV (All Terrain Vehicle); a pickup truck; a heavy-duty transport vehicle, such as a bulldozer, a mobile crane and an earth-moving vehicle; an airplane; a boat; a ship; and a part of other modes of transport. Particularly preferably, the term "automobile part" refers to a part of a car; a truck; a semi-trailer truck; a tractor; a motorcycle; a trailer; an ATV (All Terrain Vehicle); a pickup truck or a heavy-duty transport vehicle. Preferably, the auto part is a body part, in particular a hood, a fender, a door, a bumper, a rear quarter panel, a trunk lid or a hatch.

[0067] The space substrate can be an uncoated space substrate, that is, a space substrate that does not contain any coating, or it can be an at least partially coated space substrate, that is, a space substrate in which at least a portion of the surface of the substrate already contains at least one coating (such as a dried or cured coating). Suitable space substrates include (i) uncoated or at least partially coated space metal substrates; (ii) uncoated or at least partially coated space plastic substrates; and (iii) uncoated or at least partially coated space substrates containing metal and plastic parts. Suitable metal substrates are selected from the group including or consisting of: steel, iron, aluminum, copper, zinc and magnesium substrates and substrates made of alloys of steel, iron, aluminum, copper, zinc and magnesium. The metal substrate can be pretreated in a manner known per se (i.e., for example, cleaned) and / or provided with a known conversion coating. Cleaning can be performed mechanically, for example, by wiping, grinding and / or polishing, and / or chemically by an etching method, such as surface etching in an acid bath or an alkaline bath using, for example, hydrochloric acid or sulfuric acid, or by cleaning with an organic solvent or an aqueous detergent. Pretreatment can be carried out by applying a conversion coating, in particular by phosphating and / or chromating, preferably phosphating. Preferably, the space metal substrate is at least treated with a conversion coating, in particular preferably phosphated by zinc phosphate treatment. Preferred space plastic substrates are substrates comprising or consisting of: (i) polar plastics, such as polycarbonate, polyamide, polystyrene, styrene copolymers, polyesters, polyphenylene ethers and blends of these plastics, (ii) synthetic resins such as polyurethane RIM, SMC, BMC and (iii) polyolefin substrates of the polyethylene and polypropylene types with a high rubber content, such as PP-EPDM, and surface-activated polyolefin substrates. In addition, the space plastic substrate can be fiber-reinforced, in particular using carbon fibers and / or metal fibers.

[0068] The (multiple) coating materials for coating at least a portion of the surface of the space substrate can be any suitable (multiple) liquid or solid coating materials, such as (multiple) primer-surfacer coating materials, (multiple) primer coating materials, (multiple) color paint materials, (multiple) varnish materials, (multiple) topcoat materials or (multiple) single-stage materials. "Primer-surfacer coating materials" (also expressed as filler coating materials) refer to coating materials for preparing intermediate layers, which are designed to fill the irregularities of the surface of the space substrate, support corrosion resistance and adhesion, and provide protection from mechanical exposure (such as stone strikes). "Primer coating materials" refer to the coating materials of the first coating layer for preparing the multilayer coating on the surface of the space substrate. Primer coating materials are used to provide improved adhesion for the multilayer coating. In addition, primer coating materials produce coatings (for example, on metal space substrates) that can provide improved anti-corrosion protection. "Color paint materials" refer to the intermediate coating materials commonly used in automobile painting to impart color. Color paint materials can be formulated as effect coating materials or solid color coating materials. Effect coating materials usually contain at least one effect pigment and optionally contain other colored pigments or spheres that give the desired color and effect, while solid coating materials contain only colored pigments and do not contain any effect pigments. "Varnish material" refers to a transparent coating material. "Transparent" means that the film formed by the coating material is not completely opaque, but has a certain degree of transparency, which allows the color of the following (multiple) coatings to be visible through the varnish layer formed by the varnish material. Therefore, the varnish material can be completely free of pigment, contain only transparent pigments, or contain pigments in an amount that does not make the coating produced by the varnish material opaque. Usually, the varnish material is applied on top of the color varnish layer formed by (multiple) color varnish materials to protect the color varnish layer below. The term "topcoat material" refers to a coating material that is applied as the last coating material in the coating process, so that the coating formed by the material is the topmost coating of the coating. The topcoat material can be a colored coating material, that is, a coating material containing effect and / or color pigments, or it can be a varnish material. The term "single-stage materials" refers to colored or transparent coating materials that are applied in a single layer on a space substrate, i.e., they do not require the application of an additional coating material on top of said layer. Robot path data can be generated for one coating material or for multiple coating materials, depending on whether the resulting coating should be a single-layer coating or a multi-layer coating and depending on the coating(s) already present on the space substrate.

[0069] On the one hand, the robot is located in the spray booth, and each spatial substrate is located in the robot's workspace. The term "robot's workspace" should be understood in a broad sense in this case, and relates to a collection of all positions that the robot, particularly a movable robot component, can reach. The robot's workspace usually depends on multiple factors, including the size of the movable robot component (such as a robot arm). Particularly preferably, the robot is mounted on a track system (for example, a Gail (gudel) track system mounted on the side and / or ceiling of the spray booth) to increase the robot's workspace and allow the robot to move along (multiple) spatial substrates to allow coating (multiple) with larger size spatial substrates. In one example, the robot's workspace corresponds to the size of the spray booth. In another example, the robot's workspace is smaller than the size of the spray booth. The spray booth can include markings for indicating different zones. This can allow the user to position (multiple) spatial substrates in the corresponding zones in the spray booth or in the corresponding areas of the zones.

[0070] On the one hand, the coating tool includes a coating material applicator, particularly a spray applicator. In this example, the coating tool further includes a coating material reservoir configured to accommodate a specific coating material, which is attached to the coating material applicator. In this example, the coating material reservoir is directly attached to the material applicator. Direct attachment means that the reservoir is directly connected to the material applicator, for example, by using an appropriate connection. This avoids the use of additional pipes present in the spray chamber that need to be considered during the generation of robot path data. In another example, the reservoir is attached to the material applicator via a pipe. In this example, the reservoir can be located inside or outside the spray chamber and is connected to the material applicator via a pipe. The coating tool can be permanently or temporarily attached to the robot. In the case where the coating tool is temporarily attached to the robot, a plurality of different coating tools (such as a coating tool including a coating material reservoir containing different coating materials) can be stored in a tool rack, as described later. Before coating, the robot controller can instruct the robot to select an appropriate coating tool from the tool rack, as described later.

[0071] Step (a):

[0072] In step (a) of the method of the present invention, spatial substrate data and coating material data are provided to at least one computer processor via a communication interface. The spatial substrate data includes substrate classification data and data indicating the geometry and color of each spatial substrate, in particular the surface of each spatial substrate. If more than one spatial substrate is to be coated with at least one coating material, spatial substrate data for each spatial substrate to be coated is provided in step (a). The coating material data includes data indicating the type of the at least one coating material. In one example, the coating material data further includes the order of coating materials to be applied to the spatial substrate. If more than one coating material is applied to the spatial substrate, this is preferred to ensure that the coating materials are applied in the correct order. In another example, the coating material data does not include the order of coating materials to be applied to the spatial substrate. This may be preferred if only one coating material is to be applied to (multiple) spatial substrates or if a predefined coating material order is used during the generation of tool path data.

[0073] The term "communication interface" is to be understood in the present case in a broad sense and relates to a software and / or hardware interface for establishing communication (e.g., transmitting or exchanging signals or data). A software interface may be, for example, a function call, an API. A communication interface may include a transceiver and / or a receiver. Communication may be wired or wireless. A communication interface may be based on or support one or more communication protocols. A communication protocol may be a wireless protocol, for example, a short-range communication protocol, such as or WiFi; or a long-range communication protocol, such as a cellular or mobile network, for example, a second generation cellular network ("2G"), 3G, 4G, Long Term Evolution ("LTE"), or 5G. Alternatively or additionally, the communication interface may even be based on a proprietary short-range or long-range protocol. The communication interface may support any one or more standard and / or proprietary protocols.

[0074] The term "computer processor" (hereinafter also referred to as "hardware processor") should be understood in the present case in a broad sense and refers to any logic circuit configured to perform the basic operations of a computer or system, and / or generally refers to a device configured to perform calculations or logical operations. In particular, a computer processor can be configured to process basic instructions that drive a computer or system. As an example, a computer processor may include at least one arithmetic logic unit ("ALU"), at least one floating point unit ("FPU") (such as a math coprocessor or a digital coprocessor), a plurality of registers (specifically registers configured to supply operands to the ALU and store the results of the operations), and memories (such as L1 and L2 cache memories). In particular, a computer processor may be a multi-core processor. In particular, a computer processor may be or may include a central processing unit ("CPU"). The computer processor may be a ("GPU") graphics processing unit, a ("TPU") tensor processing unit, a ("CISC") complex instruction set computing microprocessor, a reduced instruction set computing ("RISC") microprocessor, a very long instruction word ("VLIW") microprocessor, or a processor that implements other instruction sets or multiple processors that implement a combination of instruction sets. The computer processor may also be one or more special-purpose processing devices, such as an application-specific integrated circuit ("ASIC"), a field programmable gate array ("FPGA"), a complex programmable logic device ("CPLD"), a digital signal processor ("DSP"), a network processor, etc. The method of the present invention may be implemented as software in a DSP, a microcontroller, or any other auxiliary processor, or as a hardware circuit in an ASIC, CPLD, or FPGA. It should be understood that the term computer processor may also refer to one or more processing devices, such as a distributed processing device system located on multiple computer systems (e.g., cloud computing), and is not limited to a single device unless otherwise specified.

[0075] Space substrate data:

[0076] On the one hand, the data indicating the geometry and color of each spatial substrate include data representing each spatial substrate in three-dimensional space, in particular a three-dimensional point cloud of each spatial substrate, and color data of each spatial substrate. The color data may include color space data or static image data. An example of color space data is defined by L*a*b*, where L* represents luminous intensity, a* represents red / green appearance, and b* represents yellow / blue appearance. Another example of color space data is defined by L*, C*, h, where L* represents brightness, C* represents chromaticity, and h represents hue. Yet another example of color space data is defined by RGB, where R represents the red channel, G represents the green channel, and B represents the blue channel. Particularly preferably, the color space data is defined by RGB. The static image data may refer to a frame acquired by a scanning device described later.

[0077] In addition to the spatial substrate classification data and the data indicating the geometry and color of each spatial substrate, the spatial substrate data may further include a substrate ID, a substrate name, a barcode, a QR code, data about a substrate manufacturer, data about a substrate composition, substrate production data, a rule set for smoothing the surface of a 3D model generated from the data indicating the geometry and color of each spatial substrate, or a combination thereof. Associating the rule set for smoothing the surface of the 3D model with the defined spatial substrate type allows different rule sets to be applied to different substrate types, thereby allowing specific smoothing rules to be defined for each substrate type and smoothing to be performed according to the substrate type.

[0078] The spatial substrate data may be provided in a variety of ways. In one non-limiting example, providing the spatial substrate data includes

[0079] - detecting, with the at least one computer processor, a user input indicative of a substrate classification associated with each spatial substrate and a user input indicative of a location of each spatial substrate within a workspace of the robot,

[0080] - determining, using the at least one computer processor, substrate classification data for each spatial substrate and a position of each spatial substrate within a workspace of the robot based on detected user input,

[0081] - providing data of the robot's workspace to the at least one computer processor via a communication interface,

[0082] - determining, using the at least one computer processor, based on the provided data of the workspace of the robot, the collision geometry present in said workspace,

[0083] - determining, using the at least one computer processor, scanning path data for a scanning path(s) to be followed by a scanning device along the surface of each spatial substrate based on the determined position of each spatial substrate within the workspace of the robot and the determined collision geometry, and providing the determined scanning path data to the scanning device via the communication interface,

[0084] - Utilizing the at least one computer processor to generate spatial substrate data for each spatial substrate by acquiring data indicating the geometric structure and color of each spatial substrate acquired by the scanning device based on the provided scanning path data via a communication interface and combining the acquired data at least with the determined substrate classification data of each spatial substrate.

[0085] The steps of detecting user input and determining the substrate classification data and the position of each spatial substrate based on the detected user input can also be performed after any time before generating the spatial substrate data. For example, one or two steps can be performed after determining the scan path data and before generating the spatial substrate data. Detecting user input can include displaying a user interface that allows the user to select the substrate classification associated with each spatial substrate to be coated, such as by displaying a list of possible substrate classifications (such as trunk lids, hoods, fenders, etc.), or by displaying a text field and prompting the user to enter the corresponding spatial substrate classification or substrate classification ID associated with each spatial substrate to be coated, and the user interface allows the user to select the position of each spatial substrate to be coated in the robot's workspace, such as by displaying a map or image of the spray booth and prompting the user to select the position of each spatial substrate to be coated in the displayed map of the spray booth. The graphical user interface can be displayed on the screen of the display device. In one example, for example, if the display device is a tablet computer, the display device accommodates at least one computer processor. In another example, for example, if the display device is an external computer monitor, a laptop monitor, or if the display device is only used to display a graphical user interface, the display device is attached to at least one computer processor via a communication interface. User input can be detected via an interactive element such as an input device or an input / output device, particularly a mouse, keyboard, trackball, touch screen, or a combination thereof. In another example, an interactive element can be a projected area that receives user input in the form of a gesture such as a finger gesture or a hand movement.

[0086] In one example, the position of each spatial substrate in the robot's workspace includes the position of each spatial substrate in at least one zone of the spray booth containing the robot. Therefore, the spray booth is divided into at least two different zones, and the user must select the zone(s) of the spray booth in which each spatial substrate is located. The selection of at least one zone can be facilitated using a graphical user interface that displays a map or image of the spray booth divided into at least two different zones and prompts the user to select at least one zone, for example, by clicking on the corresponding zone or the number of the zone or by entering the number(s) assigned to the corresponding zone(s).

[0087] The data of the robot's workspace can be provided by obtaining the data from a data storage medium (such as an internal memory or a database connected to at least one computer processor via a communication interface), or by scanning a spray booth including (multiple) spatial substrates to be coated. In one example, the data includes data representing the workspace in three-dimensional space, in particular a three-dimensional point cloud of the workspace, and color data of the workspace. For example, a 3D point cloud of the workspace can be obtained by attaching a well-known 3D scanning tool to the robot and using the tool to scan the robot's workspace. The color data of the workspace is necessary for performing step (b) of the process of the present invention and determining the coatings that already exist on the substrate, which may require the application of certain rules contained in the coating tool parameter data to obtain a smooth transition between the already existing coating and the (multiple) coating materials applied by the robot's coating tool.

[0088] Generating collision geometry based on provided data of the robot's workspace can be performed according to methods well known in the prior art, and can include filtering the provided data to reduce the number of data points, determining whether the data contains geometry of a certain size, creating (multiple) 3D objects present in the workspace from extreme data points, and filling the generated (multiple) objects with a volume, or a combination thereof.

[0089] Since the data about the robot's workspace is only sufficient to determine the collision geometry, it is necessary to determine the scan path data to obtain detailed data about the geometry and color of each spatial substrate present in the robot's workspace to allow the generation of tool path data, as described later. In one example, the scan path data is determined for each zone of the spray booth based on the position of each spatial substrate in the robot's workspace and the determined collision geometry. If (multiple) spatial substrates are present in more than one zone of the spray booth, these zones can be traversed in a defined order when determining the scan path data. The scan path data is preferably composed of a grid-like series of data points in space, each series representing a complete travel line between the beginning of the grid stroke and the end of the grid stroke along the surface of each spatial substrate to be scanned. The determined scan path data is provided to a scanning device (hereinafter also referred to as a sensor device or sensor system) via a communication interface. In the case where the scanning device is attached to the robot, the determined scan path data is preferably provided to a robot controller connected to the robot via a communication interface, and the robot controller is configured to control the robot using the received scan path data. In the case where the scanning device is separate from the robot, the scanning path data is provided to the scanning device or a controller configured to control the scanning device. It is preferred to attach the scanning device to the robot performing the coating material application and provide the generated scanning path data to the robot controller, as this reduces the number of devices and therefore the complexity required to generate spatial substrate data and fully automatically coat the spatial substrate.

[0090] The generated spatial substrate data can be associated with the substrate ID and can be provided to the data storage medium via the communication interface for storage. This allows the generated spatial substrate data to be obtained using the substrate ID at a later point in time (for example, during the generation of tool path data or when the method of the present invention is performed for the same spatial substrate at a later point in time), thereby avoiding the generation of spatial substrate data the next time the data is needed. If there is more than one spatial substrate in the robot's workspace, the spatial substrate data generated for each spatial substrate is associated with the appropriate substrate ID. For example, this can be performed by assigning a position ID to the scan path data obtained for each spatial substrate and associating the position ID with the position of each spatial substrate determined based on the detected user input.

[0091] Coating material data:

[0092] The coating material data at least includes data indicating the identity of the coating material and may further include the order of the coating materials to be applied to each spatial substrate and / or data indicating the coating tool to be used to apply each coating material. The data indicating the identity of the coating material preferably includes the name of each coating material type, the ID of each coating material type, or a combination thereof. The name of the coating material type may include, for example, varnish, paint, sealant, primer, primer-surfacer, topcoat, single-stage material, etc. Each coating material type may be assigned a unique ID so that each coating material type is identified by its uniquely assigned ID, such as varnish, paint, sealant, primer, primer-surfacer, topcoat, single-stage material. The data indicating the coating tool may include a unique coating tool ID, a coating tool type (such as ESTA, pneumatic applicator, etc.), or a combination thereof.

[0093] For example, the coating material data may be provided by displaying a user interface and prompting a user to input data indicating the identity of the coating material(s), such as the type of the coating material(s), i.e., varnish, paint, sealant, primer, primer-surfacer, topcoat, single-stage material. The identity of the coating material may also be provided by displaying a list of existing coating material types within the user interface and detecting a user input indicating a selection of at least one of the list items. The user may also input or provide an ID / barcode / QR code for each coating material, and the data indicating the identity may then be retrieved by the processor from a data storage medium on which the coating material ID / barcode / QR code is stored in association with its corresponding data indicating the identity of the type of coating material.

[0094] In one example, the order of coating materials to be applied needs to be input by the user, for example by displaying a user interface and prompting the user to input the order of the provided coating material identities. In another example, a predefined order is used for the data provided indicating the identity of one / multiple coating materials. For example, the predefined order can be determined according to a rule set, which contains rules for determining the order of coating materials to be applied based on the provided coating material types. For example, the rule set may include a rule that determines that the user-input coating material types "paint" and "varnish" are applied in the following order: paint, followed by varnish.

[0095] In one example, data indicative of a coating tool must be input by a user, such as by displaying a user interface and prompting the user to input the data, such as by displaying a list and prompting the user to select a list item or by displaying a text field and prompting the user to enter the appropriate data. This may be preferred if the coating material can be applied with multiple coating tools, because in this case, data indicative of the type of at least one coating material cannot be used to derive application parameters associated with a specific coating tool because multiple coating tools can be used.

[0096] On the one hand, the coating material data further includes chemical property data of (multiple) coating materials, physical property data of (multiple) coating materials, data about the composition of each coating material, the ID of each coating material, the barcode of each coating material, the QR code of each coating material, data about the material manufacturer of each coating material, data indicating the space substrate to be coated with (multiple) coating materials, or a combination thereof. The data indicating the space substrate to be coated may include the substrate ID or substrate type described above. Typically, this data is only necessary when more than one space substrate is to be coated and the (multiple) coating materials used to coat the multiple space substrates are different for at least a portion of the space substrates. In this case, data indicating the space substrate to be coated with (multiple) coating materials must be provided to ensure that the correct (multiple) coating materials are applied to each space substrate.

[0097] Optional step (b):

[0098] In step (b), the computer processor determines, based on the data provided in step (a), whether each spatial substrate, in particular a portion of the surface of each spatial substrate, includes at least one masking material, which step is generally optional. The use of a masking material on a portion of the surface of the (multiple) spatial substrates allows coating of masked areas to be avoided, and for example, the masking material can be used if the (multiple) spatial substrates include areas that should not be coated, or if the (multiple) spatial substrates are to be coated with two different colors of coating materials and there is a clearly visible separation between the applied coating materials. Performing this step allows reducing the consumption of coating material and overspray, because areas covered with masking material are excluded during the determination of the robot path data, thereby avoiding the application of coating material to the masking material.

[0099] The term "shielding material" is to be understood in the present case in a broad sense and relates to a material applied to a portion of the surface of a space substrate to protect said surface from the effects of a coating material applied to the space substrate using a coating tool attached to a robot. Thus, the surface of the space substrate covered by the shielding material is not coated with the applied coating material. The shielding material should have sufficient adhesion to the surface of the space substrate to avoid removal of the shielding material during the coating process. However, the shielding material should be removable without leaving undesirable residues on the surface of the space substrate to avoid time-consuming cleaning operations after removal.

[0100] On the one hand, the masking material comprises masking paper, masking tape, masking film, or a combination thereof. For example, masking paper can be fixed to a portion of the surface of the space substrate using masking tape or masking film.

[0101] Determining whether each spatial substrate includes at least one shielding material based on the provided spatial substrate data may include generating (multiple) three-dimensional (3D) models using the provided spatial substrate data, and providing the generated (multiple) 3D models to a data-driven model parameterized according to historical data of shielding materials, in particular according to historical 3D models of spatial substrates containing (multiple) shielding materials. Preferably, the (multiple) 3D models are created using 3D point cloud data and color data contained in the provided spatial substrate data, that is, each 3D model is a 3D color model. Generating (multiple) 3D models based on point cloud data and color data can be performed according to methods well known in the prior art.

[0102] "Data-driven model" can refer to a model derived at least in part from data. Relationships that cannot be modeled by physical and chemical laws can be described using data-driven models. Relationships that do not need to be solved according to physical and chemical laws can be described using data-driven models. This can reduce computing power and can increase speed. Data-driven models can be obtained from machine learning (Machine Learning and Deep Learning frameworks and libraries for large-scale data mining: a survey [Machine Learning and Deep Learning frameworks and libraries for large-scale data mining: a survey], Artificial Intelligence Review [Artificial Intelligence Review], Vol. 52, 2019, pp. 77-124). Data-driven models can include empirical models or so-called "black box models". An empirical or "black box" model can refer to a model constructed by using one or more of machine learning, deep learning, neural networks, or other forms of artificial intelligence. An empirical model or "black box" model can be any model that produces a good fit between training data and test data.

[0103] In one example, the data-driven model is a trained machine learning model that has been trained to determine the shielding materials present on the surface of the space substrate based on the historical color data of the shielding materials present on the surface of the historical space substrate. The historical color data may include a 3D color model of the space substrate, each space substrate including at least one shielding material. The trained machine learning module may be a classifier that classifies which surface areas of the provided 3D color model contain (multiple) shielding materials. "Machine learning" may refer to a computer algorithm that continuously improves through experience and builds a model using supervised, unsupervised or semi-supervised machine learning techniques based on sample data (usually described as training data). Supervised learning includes using training data with known labels or results and preparing the model through a training process, during which the model needs to make predictions and be corrected when these predictions are wrong. The training process will continue until the model reaches the desired level of accuracy on the training data. Semi-supervised learning includes using a mixture of labeled input data and unlabeled input data and preparing the model through a training process, during which the model must learn structures to organize data and make predictions. Unsupervised learning involves using unlabeled input data with no known outcomes and preparing a model by inferring the structures present in the input data (such as general rules, similarities, etc.).

[0104] In one example, a machine learning algorithm is trained by selecting inputs and outputs to define the internal structure of the machine learning algorithm, applying a collection of input and output data samples to train the machine learning algorithm, verifying the accuracy of the machine learning algorithm by applying input data samples that include (multiple) known shielding materials, comparing the generated output values ​​with the expected output values, and modifying the parameters of the machine learning algorithm using an optimization algorithm in the event that the received output values ​​do not correspond to the input data samples. As input, previously described images of historical spatial substrates that include (multiple) shielding materials or do not include (multiple) shielding materials can be used. The input data is randomly selected, but only if the training data contains a complete spectrum of shielding materials. The output can be a 3D model of the spatial substrate indicating areas that include shielding materials, or it can be a classifier in the event that the spatial substrate does not include any shielding materials.

[0105] In principle, one skilled in the art can select an appropriate machine learning model or algorithm taking into account preprocessing, the existence of a solution set, the distinction between regression and classification problems, computational load, and other factors. A machine learning algorithm cheat sheet can be used for this purpose (see P. Sivasothy et al.: "Proof of concept: Machine learning based filling level estimation for bulk solid silos"; Proc. Mtgs. Acoust.; Vol. 35; 055002; 2018). Figure 6). In the present invention, the machine learning algorithm may be (i) a deep learning algorithm, such as a long short-term memory (LSTM) algorithm, a gated recurrent unit (GRU) algorithm, or a perceptron algorithm, (ii) an instance-based algorithm, such as a support vector machine (SVM), in particular a deep learning algorithm. "Deep learning" may refer to methods based on artificial neural networks (ANNs) having an infinite number of bounded-size layers, which allow for practical applications and optimized implementations while maintaining theoretical generality under mild conditions. The deep learning architecture implementing the deep learning algorithm may include a deep neural network, a deep belief network (DBN), a recurrent neural network (RNN), and a convolutional neural network (CNN). Suitable optimization algorithms for manipulating the parameters of (multiple) learning algorithms during training are known in the art and include, for example, gradient descent, momentum, rmsprop, Newton-based optimizers, adam, BFGS, or model-specific methods. These optimization algorithms are used during training of the machine learning algorithm to modify the parameters in each training step, thereby reducing the difference between the output of the machine learning algorithm and the expected output until a predefined termination criterion, such as the number of iterations or accuracy, is obtained.

[0106] The determination of the presence or absence of a masking material by the trained machine learning model can be provided to a display device for display on a screen. For example, (multiple) 3D color models including a classification of which surface areas of each spatial substrate contain (multiple) masking materials can be provided to a display device for display on a screen (e.g., within a graphical user interface). This allows a user to check whether the presence and position of (multiple) masking materials are correctly detected by the trained machine learning model, and if the (multiple) masking materials present on the (multiple) spatial substrates are not correctly determined, their presence and position are corrected. The results of the determination and data associated with the user's approval or (multiple) corrections can be provided to a computing device used to train the machine learning model via a communication interface as a training data set. This allows the performance of the trained machine learning model to be improved using data acquired during the method of the present invention.

[0107] Step (c):

[0108] In the step (c) of the inventive method, various data, i.e. coating tool parameter data, coating program data and substrate type data, are obtained via a communication interface using at least one computer processor. The data can be stored on a data storage medium (such as an internal memory or a database connected to at least one computer processor via a communication interface), and can be obtained, for example, using the provided spatial substrate data and the substrate classification data and coating material ID contained in the coating material data. "Data storage medium" can refer to physical and other computer-readable media for carrying or storing computer executable instructions and / or data structures. Such a computer-readable medium can be any available medium that can be accessed by a general or special computer system. Computer-readable media can include physical storage media for storing computer executable instructions and / or data structures. Physical storage media include computer hardware, such as RAM, ROM, EEPROM, solid-state drive ("SSD"), flash memory, phase change memory ("PCM"), optical disk storage, disk storage or other magnetic storage devices, or any other (multiple) hardware storage devices that can be used to store program codes in the form of computer executable instructions or data structures, and the (these) hardware storage devices can be accessed and executed by a general or special computer system to implement the functions disclosed in the present invention. "Database" may refer to a collection of related information that can be searched and retrieved. A database may be a searchable electronic digital, alphanumeric, or text document; a searchable PDF document; a Microsoft electronic spreadsheet; or a database as known in the art. A database may be a set of electronic documents, photos, images, charts, data or drawings residing in a computer-readable storage medium that can be searched and retrieved. A database may be a single database or a set of related databases or a set of unrelated databases. "Related databases" means that there is at least one common information element in the related databases that can be used to relate the databases.

[0109] Coating tool parameter data:

[0110] The coating tool parameter data includes coating tool tolerance data and at least one application parameter associated with the coating tool. The term "application parameter associated with the coating tool" refers to parameters for applying a specific liquid or solid coating material using a defined coating tool. For example, if the coating tool is a spray applicator and the coating material is a liquid paint material, the parameters refer to the spray parameters necessary to apply the specific liquid paint material coating material to the surface of the space substrate using the spray applicator.

[0111] On the one hand, the coating tool tolerance data includes target distance data, overlap percentage data, pattern size data, (multiple) rotation tolerances around the z-axis of the coating tool, (multiple) rotation tolerances around the x-axis of the coating tool, (multiple) rotation tolerances around the y-axis of the coating tool, or a combination thereof. The target distance data and the overlap percentage may refer to a defined target distance value / overlap percentage and / or a target distance range / overlap percentage range. The combination of a defined value and a range allows the use of the defined value as a starting point for generating tool path data, and if tool path data with the required quality needs to be generated, the starting point may be adjusted according to the provided range. The term "target distance" refers to the distance from the end of a coating tool (such as a nozzle of a coating material applicator) to the surface of a space substrate, and may be in the range of from 6 inches to 12 inches. The pattern size may be fixed for a specific target distance and may be scaled with the target distance. For example, a defined pattern size may be associated with a target distance of 6 inches and a target distance of 12 inches, and a linear interpolation may be used to determine the pattern size of the target distance within the range. The pattern size data may therefore include rules that associate a given target distance with a particular pattern size or range of pattern sizes.

[0112] On the one hand, at least one application parameter associated with the coating tool includes the flow rate of the coating material, the voltage applied to the coating material, the pressure applied to the coating material, the bell speed of the coating tool, or a combination thereof. The application parameters included in the coating tool parameter data depend on the type of coating tool, so different coating tools will result in different application parameters included in the coating tool parameter data. For example, an electrostatic spray applicator requires a voltage parameter, which is not necessary for a pneumatic spray applicator.

[0113] Based on the provided coating material data, coating tool parameter data is obtained using at least one computer processor. Based on the provided coating material data, obtaining coating tool parameter data may include obtaining data indicating the type of at least one coating material contained in the provided coating material data and optionally data indicating a coating tool; and obtaining coating tool parameter data associated with the obtained data indicating the type of at least one coating material and optionally the obtained data indicating the coating tool. The coating tool parameter data may be stored on a data storage medium and may be associated with data indicating the type of the coating material (such as the previously mentioned unique ID). The unique ID may be used to obtain appropriate coating tool parameter data from a data storage medium (such as an internal memory or a database connected to at least one computer processor via a communication interface).

[0114] Coating procedure data:

[0115] The coating program data obtained include rule sets for coating outer edges and edges adjacent to (multiple) open spaces and rule sets for coating major surfaces. The term "rule set" refers to an execution unit of a rule and / or a decision table and / or an algorithm, that is, a rule set organizes rules and / or decision tables and / or algorithms into execution units. Multiple rule sets (also called rule streams) can be executed in a defined order. In a rule set, rule priorities can be used to specify the order in which the rules contained in the rule set should be executed. Each rule set includes one or more rules. Each rule contains a condition and an action associated with the condition, that is, an action that is executed if the condition is determined to be met or true.

[0116] Each rule set may include one or more rules or algorithms. For example, a rule set for coating the outer edge of a space substrate and the edge adjacent to (multiple) open spaces may include at least one algorithm for determining the outer edge and at least one algorithm for determining the edge adjacent to (multiple) open spaces present in the surface space substrate. The algorithm for determining the edge adjacent to (multiple) open spaces may also allow for determining (multiple) open spaces present in the space substrate, which are at least partially surrounded by the determined edge adjacent to (multiple) open spaces. "Edge adjacent to (multiple) open spaces" refers to an edge that exists adjacent to (multiple) open spaces in the surface of the space substrate. For example, such an open space can be represented by a marker light recess, a window frame, a headlight recess, etc. "(Multiple) open spaces" refers to holes present in the space substrate, such as marker light recesses on fenders, windows on car doors, and hood air intake recesses on the hood, as well as indentations in the space substrate, such as headlight recesses in the fender, the top inner rail of the fender and the lower flange on the tab or fender dogleg.

[0117] On the one hand, the rule set for coating the outer edge and the edge adjacent to the (multiple) open space includes the rule for coating the edge adjacent to the (multiple) open space, the rule for coating the outer edge, and the optional rule for coating the (multiple) open space present in the space substrate, which is at least partially surrounded by the edge adjacent to the (multiple) open space. At least the rule for coating the edge adjacent to the (multiple) open space and the rule for coating the (multiple) open space present in the space substrate may include coating tool tolerance data different from the coating tool tolerance data included in the obtained coating tool parameter data. In one example, the rule for coating the outer edge may also include coating tool tolerance data different from the coating tool tolerance data included in the obtained coating tool parameter data. For example, for a space substrate including a grille (such as a car bumper cover), this may be the case. Therefore, the target distance data, overlap percentage data, pattern size data and (multiple) rotation tolerances included in the rules are different from the target distance data, overlap percentage data, pattern size data and (multiple) rotation tolerances included in the coating tool parameter data. This allows modification of the derived coating tool tolerance data in the event that an edge adjacent to an open space(s) is detected and thus also the open space(s) is detected, and thereby allows the open space(s) and its edges to be coated using different target distances and / or overlap percentages and / or pattern sizes and / or rotational tolerances. For example, the coating tool may be allowed to vary in angle on the x-axis (i.e., pitch) by no more than 45° from parallel to the surface plane to coat the open space(s) and / or the edges adjacent to the open space(s).

[0118] On the one hand, the rule set for coating the main surface comprises the rule for determining that the coating program starts, the rule of coating direction, the rule that the coating tool rotates in the tool path, the rule of surface separation or its combination." main surface " refers to the surface between the outer edge present in the space substrate, does not include the open space in the described surface. Therefore, the main surface may not include any open space, and may also include open space. However, the open space existing in the main surface is not counted in the main surface.

[0119] By using a rule set during the generation of robot path data, such robot path data can be reliably generated for substrates with large geometric variations. This allows such substrates to be coated by a robotic system using the generated robot path data in order to obtain a coating with high optical and mechanical quality, regardless of the geometry of the respective substrate. Thus, such a rule set can be used to customize the robot path data to the respective substrate without requiring manual interaction to ensure that the geometry of the substrate is fully considered during the generation of the robot path data, thereby avoiding inappropriate coating material application that may result in reduced optical and / or mechanical quality of the resulting coating.

[0120] If at least two different coating materials are applied, the above-mentioned coating tool parameter data and coating program data are obtained for each coating material, and tool path data are generated for each coating material using the obtained coating tool parameter data, coating program data and the obtained spatial substrate data, as described below.

[0121] Substrate Type Data:

[0122] The substrate type data is obtained based on the provided spatial substrate data, in particular based on the substrate classification data contained in the provided spatial substrate data. The obtained substrate type data includes at least one rule set for the spatial substrate type that matches the substrate classification data contained in the provided spatial substrate data. The term "spatial substrate type that matches the substrate classification data" refers to a spatial substrate type that is the same as the previously mentioned substrate classification data or is a spatial substrate type associated with a unique classification ID. For example, if the substrate classification data is equal to a fender, the type of the spatial substrate is also a fender. Similarly, if the substrate classification data is equal to a unique classification ID, the type of the spatial substrate corresponds to the type associated with the unique ID. If different types of spatial substrates are to be coated, substrate type data is obtained for each spatial substrate classification data contained in the spatial substrate data provided for each spatial substrate in step (a).

[0123] In one aspect, the at least one rule set includes at least one rule for coating edges adjacent to (multiple) open spaces of a corresponding type of spatial substrate, data on the required quality of the (multiple) tool paths, optionally at least one rule for coating (multiple) open spaces present within the spatial substrate, and optionally at least one rotational tolerance of the coating tool. The (multiple) rotational tolerances of the coating tool may include the previously mentioned rotational tolerances around the x-axis, y-axis, and z-axis of the coating tool.

[0124] Retrieving substrate type data may include determining substrate classification data included in the provided spatial substrate data and retrieving substrate type data associated with the determined substrate classification data from a data storage medium. To this end, the substrate type data may be associated with the substrate classification data before storing the data on a data storage medium, such as an internal memory or a database connected to at least one computer processor via a communication interface.

[0125] Step (d):

[0126] In step (d) of the method of the present invention, at least one computer processor is used to generate tool path data for (multiple) tool paths to be followed by the coating tool along the surface of each spatial substrate based on the data obtained in step (c) and the results of the determination performed in step (b) optionally. "Tool path" refers to the movement path of the coating tool, in particular a coating material applicator (such as a spray applicator) relative to the surface of each spatial substrate, in particular relative to the surface of each spatial substrate to be coated with the coating material applied from the coating tool (such as a coating material applicator). Therefore, the tool path includes a trajectory of the three-dimensional position and orientation angle of the distal end of the operating axis, which follows the topographical contour of each spatial substrate with a certain offset to avoid collision between the coating tool and each spatial substrate.

[0127] In one aspect, generating tool path data for the coating tool to follow along the surface of each spatial substrate includes:

[0128] - generating, with the at least one computer processor, a 3D model of each spatial substrate based on the provided spatial substrate data, and optionally applying a set of rules to smooth the surface of each generated 3D model,

[0129] - using the computer processor to determine, based on the acquired coating process data and the generated and optionally smoothed 3D model(s), the outer edge(s), the edge(s) adjacent to the open spaces, the major surface, and the open spaces(s) present within each spatial substrate,

[0130] - generating, with the at least one computer processor, tool path data for outer edges and tool path data for edges adjacent to the open space(s) based on the obtained coating program data, the obtained coating tool parameter data, the obtained substrate type data, the determined outer edge(s), edge(s) adjacent to the open space(s) and the open space(s) and the generated and optionally smoothed 3D model(s),

[0131] - generating, with the at least one computer processor, tool path data for the major surface of each spatial substrate based on the acquired coating procedure data, the acquired coating tool parameter data, the determined major surface and the generated and optionally smoothed 3D model(s), and

[0132] - Optionally, repeating said steps for at least one further coating material based on the acquired coating program data and the acquired coating tool parameter data associated with the at least one further coating material.

[0133] In one example, at least one computer processor is used to generate a 3D (3-dimensional) model of each spatial substrate using the 3D point cloud data contained in the spatial substrate data provided in step (a). Each 3D model can be generated from the point cloud data using commonly available computer software (such as the open source library Open3D). At least one rule set can be used to smooth each generated model. The rule set can include rules that allow the determination of the surface that can be smoothed for each 3D model and the degree of smoothness associated with the surface. Preferably, the degree of smoothness is selected so that the information required for generating the tool path data is not lost after the smoothing operation. Since the degree of smoothness of the surface allows the quality of the resulting coating to be controlled, that is, the higher degree of smoothness of the surface will result in a lower overall quality in terms of the optical appearance of the coated surface, the degree of smoothness of the surface allows the overall optical appearance of the resulting surface to be controlled. For surfaces that do not require high quality in terms of optical appearance (such as headlight recesses present in fenders), a higher degree of smoothness can be used on the generated 3D model than surfaces that require high quality in terms of optical appearance (such as the part of the fender visible after the coated fender is mounted to the vehicle body). Smoothing each 3D model allows more efficient generation of tool path data for each spatial substrate and coating material to be applied to the (multiple) spatial substrates because the mesh of each 3D model obtained from the corresponding point cloud data is converted into surfaces, which can be more efficiently processed during the generation of the tool path data, for example by (multiple) computer processors that generate the tool path data, as described later.

[0134] Generating, using at least one computer processor, tool path data for outer edges and tool path data for edges adjacent to the open space(s) based on the acquired coating procedure data, the acquired substrate type data, and the generated and optionally smoothed 3D model may include

[0135] - determining target distance(s), overlap percentage(s), pattern size and rotational tolerance(s) of the coating tool based on the determined outer edges, the determined edges adjacent to the open space(s), the determined open space(s), the acquired coating tool parameter data, the acquired coating procedure data, the acquired spatial substrate data and the generated and optionally smoothed 3D model, and

[0136] - Using the generated and optionally smoothed 3D model, tool path data is generated according to the determined target distance(s), overlap percentage(s), pattern size and rotational tolerance(s) of the coating tool.

[0137] Generating, with at least one computer processor, tool path data for the primary surface based on the acquired coating program data, in particular a set of rules for coating the primary surface, the acquired coating tool parameter data and the generated and optionally smoothed 3D model may include:

[0138] - determining, based on the acquired coating process data and the generated and optionally smoothed 3D model(s), whether the main surface of each spatial substrate comprises at least two independent surfaces,

[0139] - determining whether a major surface of each spatial substrate comprises a convex shape, and creating a shell around the spatial substrate(s) using a convex hull method,

[0140] - determining target distance(s), overlap percentage(s), pattern size and rotation tolerance(s) of the coating tool based on the determined independent surfaces, the determined convex shape, the derived coating tool parameter data, the derived coating procedure data and the generated and optionally smoothed 3D model(s), and

[0141] - Using the generated and optionally smoothed 3D model(s), tool path data is generated according to the determined target distance(s), overlap percentage(s), pattern size, rotational tolerance(s) of the coating tool.

[0142] In this example, the presence of independent surfaces is determined by determining whether each space substrate includes a surface with a certain relative angle and radius of curvature between the two resulting faces. For example, if the determined relative angle is from 25° to 90° and the radius of curvature is from 1 inch to 10 inches, the separation of the surfaces is given. Ensuring that the ranges of relative angles and radii of curvature are met can ensure that small body lines with small radii of curvature but not much difference in angle between the two resulting faces are determined as independent surfaces, thereby avoiding unnecessary separation of space substrate surfaces.

[0143] For example, a known convex hull algorithm capable of calculating the convex hull of a 3D spatial substrate may be used to determine a shell around a convex surface of a (multiple) spatial substrate (such as a convex surface of a bumper cover). The shell may then be used to generate tool path data for the surface.

[0144] In one example, generating tool path data for the tool path (s) to be followed by the coating tool along the surface of each spatial substrate further includes using the generated tool path data to calculate the corresponding coating material application, and determining whether the coating produced by the calculated coating material application meets at least one predefined parameter. In the case where the coating does not meet at least one predefined parameter, at least one computer processor can use the determined result to optimize the generated tool path data. This allows ensuring that the coating produced by applying the coating material using the robot path data generated from the tool path data meets predefined quality parameters, such as wet film and / or dry film thickness and the surface area to be coated.

[0145] The predefined parameters may include, for example, wet and / or dry film thickness or ranges thereof and / or surface area to be coated or ranges thereof.

[0146] Calculating the coating material application using the generated tool path data can include simulating the coating material application using the generated tool path data and a specific applicator for the corresponding coating, such as by using a shader that simulates coating coverage by projecting light onto the surface of the 3D model. The simulation allows the generated and optionally smoothed 3D model to be calculated using predefined parameters, which are the amount of material deposited by the simulation. The coverage map can be used to determine whether a specific target area of ​​the 3D model of each space substrate will be coated with an appropriate percentage of coating material so that the wet film and / or dry film thickness of the resulting coating meets the predefined range. The coverage map can also be used to control the thickness of the transparent coating material, or control the coating thickness on the part of the (multiple) space substrate that requires a specific specification for transmittance, such as ADAS (Advanced Driver Assistance System) and radar sensor. The calculated coverage map can be provided to a display device, for example, via a communication interface to be displayed on the device. The coverage map and further calculated data can be displayed in a graphical user interface (GUI). The coverage map can include different colors to indicate whether (multiple) predefined parameters are met.

[0147] Step (e):

[0148] In step (e) of the method of the present invention, at least one computer processor generates robot path data based on the tool path data generated in step (d).

[0149] Generating (multiple) robot paths may include determining a collision geometry within the robot's workspace based on the spatial substrate data, and determining the robot path data based on the determined collision geometry and the generated tool path data. The determination of the collision geometry ensures that the robot does not collide with and damage each spatial substrate during the coating procedure. The robot path data is generated so that the robot does not collide with each spatial substrate present in its workspace and the coating tool can follow the generated tool path data. This may include determining an initial pose of the robot, determining all subsequent robot movements that follow the generated tool path data, and determining a return to the initial pose.

[0150] In one aspect, generating robot path data further comprises

[0151] - ordering, using the at least one computer processor and prior to generating the robot path data, the generated tool path data such that the robot path(s) generated from the tool path data for the outer edges and edges adjacent to the open space(s) are executed before or after the robot path(s) generated from the tool path data for the primary surface, and / or

[0152] - Optimizing the generated or sequenced robot path data using the at least one computer processor.

[0153] It may be beneficial to perform the robot path(s) for the outer edges and the edges adjacent to the open spaces before the robot path(s) for the main surface, since the coating of the edges will produce overspray on the main surface, which overspray can then be covered by coating the main surface, thereby avoiding a negative impact on the final overall appearance. In addition, this avoids applying too much coating material on the edges, which is disadvantageous because the presence of too much coating material on the edges is prone to sagging, dripping and heavy edges, thereby negatively affecting the final overall appearance of the coated substrate. However, it is also possible to perform the robot path(s) for the main surface first and then the robot path(s) for the edges. In this case, the application parameters must be adapted so that the overspray resulting from coating the edges is hidden and does not negatively affect the resulting overall appearance.

[0154] Optimizing the generated or sequenced robot path data may be beneficial because it allows smoothing the robot's motion to make it more efficient and consistent. The optimization of the generated or sequenced robot path data can be based on open source libraries such as Descartes (ROS-Industrial project, used to perform path planning on undefined Cartesian trajectories) and software frameworks such as trajopt (a software framework for generating robot trajectories through local optimization).

[0155] Step (f):

[0156] In step (f) of the method of the present invention, the generated robot path data is provided via a communication interface. In the case where the robot path data is sorted and / or optimized, the sorted or optimized robot path data is provided via a communication interface.

[0157] In one example, this includes providing the generated robot path data to a robot controller connected to the robot including the coating tool via a communication interface, as described later.

[0158] In another example, this includes providing the generated robot path data to a display device for display on the device. This may include simulating the robot path data to visualize the robot movement so that a user can ensure that the determined robot path data will not result in significant damage to the spatial substrate or the robot during the coating process. The user may have to approve the generated robot path data before providing the data to the robot controller.

[0159] In yet another example, the generated robot path data is provided to a data storage medium via a communication interface. To this end, the generated robot path data can be correlated with the spatial substrate ID to allow the generated robot path data to be retrieved when the same spatial substrate is to be coated again, thereby reducing the amount of time required to generate the required robot path data.

[0160] Further steps:

[0161] In addition to steps (a) to (f), the method of the present invention may also include further steps. In one aspect, the method of the present invention further includes the following steps: using at least one computer processor to determine the amount of each coating material necessary for coating each space substrate based on the generated tool path data and the provided coating material data. The amount of each coating material can be calculated based on the flow rate contained in the obtained coating tool parameter data and based on the generated tool path data, in particular the data indicating the application duration (i.e., the data associated with the duration of opening the coating tool during the coating process of each space substrate). This further step can be performed before step (e), before step (f), or after step (f). The determined amount of each coating material can be provided to a display device (e.g., within a GUI) via a communication interface for display, and / or provided to a mixer, thereby allowing the corresponding amount of coating material to be automatically prepared. Executing this further step allows the user to obtain information about the amount of coating material required for the coating process and thereby allows the use of this data for inventory planning or determination of the amount of coating material required for the coating program for each space substrate. The latter avoids preparing more coating material than required for the coating program, thereby allowing the reduction of waste and the cost associated with waste coating materials. Furthermore, the determined amount and information of the coating material contained in the coating material data (such as the coating material ID) can be provided to an automatic mixer, which then mixes the determined amount based on the received data. This allows the coating process to be fully automated and avoids waste coating material produced due to mixing errors.

[0162] At least one computer processor that performs steps (d) to (f) may be the same computer processor or may be a different computer processor, i.e., the computer processor that performs step (d) may be different from the computer processor that performs steps (e) and (f). In one example, the computer processor that performs steps (d) to (f) may be located on a server so that steps (d) to (f) are performed in a cloud computing environment and may be connected to another computing device that acts as a client device. A "client device" may refer to a computer or program that, as part of its operation, relies on sending a request to another program or computer hardware or software that accesses a service provided by a server. Preferably, the server may be an HTTP server and accessed via conventional web-based Internet technology. The client device provides the spatial substrate data and the data obtained in step (c) to the server via a communication interface. This allows the use of client devices with lower computing power because resource-intensive calculations are performed on the server. Internet-based systems are particularly useful if a service for generating robot path data is provided to a client via the Internet.

[0163] Embodiments of the computing device of the present invention

[0164] In one aspect, the computing device of the present invention further comprises at least one database comprising spatial substrate data, coating parameter data, coating procedure data, substrate type data, coating material data, or a combination thereof.

[0165] In one example, the above data may be stored in a single database. In another example, at least a portion of the spatial substrate data, coating parameter data, coating procedure data, substrate type data, and coating material data may be stored in the same database, i.e., the database may include, for example, spatial substrate data and coating material data. In yet another example, the above data may each be stored in a separate database.

[0166] Embodiments of the robot system of the present invention

[0167] The robotic system may include a computing device of the present invention for generating robotic path data and a robotic device configured to receive the generated robotic path data and use the received robotic path data to apply at least one coating material to at least a portion of the surface of each spatial substrate. The robotic system may include a computing device for generating robotic path data, the computing device including at least one processor and a memory storing instructions that, when executed by the processor, configure the device to perform the steps of the method disclosed herein.

[0168] In one aspect, the robotic device includes at least one movable robotic member, the at least one movable robotic member including a coating tool and a robotic controller, the robotic controller being adapted to receive the generated robotic path data from the device and being operable to move the at least one movable robotic member according to the received robotic path data to coat at least a portion of the surface of each spatial substrate with at least one coating material. The coating tool is preferably a coating material applicator, such as a spray applicator typically used to apply a liquid or solid coating material to at least a portion of the surface of the substrate. The at least one movable robotic member can be a movable robotic arm configured to include a coating tool.

[0169] In one aspect, the robot device further includes a sensor system configured to generate data of the robot's workspace and / or data indicating the geometry and color of each space substrate, and provide the generated data to the computing device of the present invention. The sensor system is preferably also used to obtain the geometry and color data of each space substrate based on the scan path data provided to the robot controller. The term "comprises" includes a permanently attached sensor system and a temporarily attached sensor system. The sensor system is preferably attached to at least one movable robot component of the robot device. In one example, the robot device includes a movable robot component, which includes a coating tool and a sensor system. In another example, the robot device includes: a first movable robot component, which includes a coating tool; and a second movable robot component, which includes a sensor system. In yet another example, the robot device includes only one movable robot component, which includes a coating tool or a sensor system, that is, the coating tool and the sensor system are only temporarily attached to the movable robot component. The temporary attachment of the sensor system can help avoid applying coating materials to the sensor system, which may cause performance degradation or failure of the sensor system. As described later, a tool changer may be used to switch between the coating tool and the sensor system.

[0170] The sensor system may include a depth sensor and may be selected from one of a camera system and a laser scanner, such as a laser scanner commercially available from Framos GmbH. The camera system or the laser scanner must be able to acquire color data and data about the geometry of the robot's workspace and / or the spatial substrate, such as 3D point cloud data.

[0171] In one aspect, the robotic system comprises a plurality of coating tools, each coating tool comprising at least one coating material reservoir containing different coating materials and connectable to a robotic device, in particular a movable robotic member. The robotic device is preferably configured to select an appropriate coating tool from the plurality of coating tools based on coating material data received from a computing device. The coating tool may include a code, a barcode or a unique ID that allows the robotic device to identify the coating tool to be attached, and the robotic device may be configured to identify an appropriate coating tool based on the code, barcode or unique ID attached to the tool and attach the appropriate coating tool to the robotic device, in particular a movable robotic member.

[0172] The robot system is preferably installed in a spray booth, which further includes a heating device. The heating device can be used to dry and / or cure the applied (multiple) coating materials to form a cured coating on each spatial substrate. Compared with a dried coating, a dried coating remains sticky and undergoes further property changes when using curing conditions (such as heating), and a cured coating is a solid coating that does not undergo any further property changes when exposed to curing conditions. The heating device can be controlled by a computing device of the system of the present invention, which can be configured to control the heating device in the following manner: based on the provided coating material data, a drying and / or curing temperature and a drying and / or curing duration are obtained (for example, by obtaining (multiple) appropriate drying and / or curing temperatures from a database using a coating material type ID or a coating material ID contained in the data), and the obtained (multiple) temperatures and durations are provided to the heating device. After the robot controller provides a signal to the computing device indicating the end of the coating procedure or the end of the application of a particular coating material, for example, after the robot device returns to the initial position after the last coating material is applied to each spatial substrate, or after the robot device returns to the tool holder to exchange the current coating tool with another coating tool containing a different coating material, the heating device can be turned on. This allows the coating and curing process to be fully automated, thereby reducing human interaction to a minimum and allowing a repeatable coating procedure to be provided despite large variations in the geometry of the substrate. BRIEF DESCRIPTION OF THE DRAWINGS

[0173] These and other features of the present invention will be more fully described in the following description of exemplary embodiments of the present invention. To facilitate identification of the discussion of any particular element or action, one or more most significant digits in the reference numerals refer to the figure number in which the element is first introduced. The description is made with reference to the accompanying drawings, in which:

[0174] Figure 1 A flow chart showing a first embodiment of the method according to the invention for generating robot path data for a robot path(s) to be followed by a robot comprising a coating tool during coating of at least a portion of a surface of a space substrate with at least one coating material,

[0175] Figure 2 A method for generating information about an object according to an embodiment of the present invention is presented. Figure 1 Block 102 depicts a flow chart of an embodiment of spatial substrate data,

[0176] Figure 3A , Figure 3B A method for generating information about an embodiment of the present invention is presented. Figure 1 Block 112 depicts a flow chart of an embodiment of tool path data,

[0177] Figure 4 The embodiment according to the present invention is shown Figure 1 A flowchart of an embodiment of block 114,

[0178] Figure 5 A flow chart showing a second embodiment of the method according to the invention for generating robot path data for a robot path(s) to be followed by a robot comprising a coating tool during coating of at least a portion of a surface of a space substrate with at least one coating material,

[0179] Figure 6 A block diagram of a computing device for generating robot path data for a robot path(s) to be followed by a robot including a coating tool during coating of at least a portion of a surface of a space substrate with at least one coating material according to an embodiment of the present invention is shown,

[0180] Figure 7 A block diagram of a robotic system for coating at least one surface of a space substrate with at least one coating material according to an embodiment of the present invention is shown,

[0181] Fig. 8A shows the z-axis of a spray applicator of a coating tool according to an embodiment of the present invention,

[0182] Figure 8B The embodiment according to the present invention is shown Fig. 8A The x-axis and y-axis of the spray applicator of the coating tool,

[0183] Fig. 9 shows a schematic diagram of a spray booth including a robot system of the present invention according to an embodiment of the present invention,

[0184] Fig.10 The embodiment according to the present invention is shown Fig. 9 Top view of the spray booth.

[0185] Fig.11 The embodiment according to the present invention is shown Fig. 9 A side view of a portion of the spray booth,

[0186] Fig. 12A The embodiment according to the present invention is shown Fig. 9 A side view of a portion of a spray booth including a tool rack containing a plurality of coating tools and a housing containing a sensor system, the coating tools and the sensor system being connectable to the spray booth using a tool changer Fig.11 The robot,

[0187] Fig. 12B Shown Fig. 12AAn enlarged view of the tool holder and housing housing the sensor system is shown. DETAILED DESCRIPTION

[0188] The specific embodiments set forth below are intended to be descriptions of various aspects of the subject matter of the present invention, and are not intended to represent the only configuration in which the subject matter of the present invention can be practiced. The accompanying drawings are incorporated herein and constitute a part of the specific embodiments. The specific embodiments include specific details for providing a thorough understanding of the subject matter of the present invention. However, it is apparent to those skilled in the art that the subject matter of the present invention can be practiced without these specific details.

[0189] In one case, the division of various parts shown in the drawings into different units can reflect the use of corresponding different physical parts and tangible parts in actual implementations. Alternatively or additionally, any single part shown in the figures can be implemented by multiple actual physical parts. Alternatively or additionally, the depiction of any two or more separate parts in the figures can reflect different functions performed by a single actual physical part.

[0190] Other figures describe these concepts in the form of flowcharts. In this form, certain operations are described as constituting different blocks that are performed in a particular order. Such implementation is illustrative and not restrictive. Certain blocks described herein may be combined and performed in a single operation, certain blocks may be separated into multiple constituent blocks, and certain blocks may be performed in an order different from that shown herein (including performing these blocks in parallel). In one embodiment, the blocks shown in the flowcharts that involve processing related functions may be composed of blocks related to Figure 6 The hardware logic circuits described herein may be implemented by one or more hardware processors and / or other logic components including a set of task-specific logic gates.

[0191] Regarding terminology, the phrase "configured to" encompasses various physical and tangible mechanisms for performing the identified operations. These mechanisms may be configured to use Figure 6 The term "logic" also encompasses various physical and tangible mechanisms for performing tasks. For example, each process-related operation shown in the flowchart corresponds to a logical component for performing that operation. The logical component can be used as described in the description of the hardware logic circuit to perform the operation. ... Figure 6 The hardware logic described herein performs its operations. When implemented by a computing device, logic components represent electrical components that are physically part of the computing system, regardless of how they are implemented.

[0192] Any storage resource or any combination of storage resources described herein can be considered a computer-readable medium. In many cases, a computer-readable medium represents a physical and tangible entity in some form. The term computer-readable medium also covers propagation signals, such as signals sent or received via physical conduits and / or air or other wireless media. However, the specific term "computer-readable storage medium" explicitly excludes propagation signals per se, but includes all other forms of computer-readable media.

[0193] The following explanation may identify one or more features as "optional." This type of statement should not be interpreted as an exhaustive indication of features that may be considered optional; that is, other features may be considered optional even though this is not explicitly stated in the text. Further, any description of a single entity is not intended to exclude the use of multiple such entities; similarly, a description of multiple entities is not intended to exclude the use of a single entity. Further, while the specification may interpret certain features as alternative ways of performing an identified function or implementing an identified mechanism, these features may also be combined together in any combination. Finally, the terms "exemplary" or "illustrative" refer to one implementation among a potential variety of implementations.

[0194] Figure 1 A flow chart of a first non-limiting embodiment of a method 100 for generating robot path data for a robot including a coating tool to follow during coating at least a portion of a surface of at least one space substrate with at least one coating material, the method being implemented by a computing device including a computer processor. The method 100 can be used to coat at least a portion of a surface of a space substrate with a coating material using a robotic system including a robot including a coating tool. The computing device can be the following with respect to Figure 6 The computing system described herein may be a fixed device (such as a fixed computer that preferably includes a computer monitor), or a mobile computing device, or may be located in a cloud computing environment. The space substrate to be coated may be an automotive part, such as an automotive body part, for example, an automotive fender. The space substrate to be coated may be an automotive hood, an automotive door, and an automotive bumper cover or any other space automotive substrate. Multiple space substrates may be coated, and these space substrates may be of the same substrate type or of different substrate types. Thus, multiple space substrates having various geometries may be coated. The automotive fender may include a conversion layer, but may not contain additional coatings. The automotive fender may already include at least one layer of primer, primer-surfacer or color paint layer. Colored color paint materials and varnish materials may be applied by a robot including a coating tool. The robot may apply more or less coating materials. The robot may be located in a spray chamber, for example, as described below with respect to Figures 7 to 11 As described.

[0195] In block 102, spatial substrate data and coating material data for each spatial substrate may be provided to a computing device implementing method 100. The provided spatial substrate data may include at least or include substrate classification data and data indicating the geometry and color of the spatial substrate, in particular the surface of the spatial substrate. In this example, the spatial substrate data includes data representing the spatial substrate in 3D space, in particular 3D point cloud data of the spatial substrate, and color data of the spatial substrate, in particular data to be used later for information about the spatial substrate. Figure 7 The RGB data or static image data acquired by the scanning device (or sensor system) described herein. In this example, the spatial substrate data further includes data indicating the identity of the substrate (such as substrate ID, substrate name, barcode, QR code), and a set of rules for smoothing the surface of the 3D model generated by the data indicating the geometry and color of the spatial substrate.

[0196] The spatial substrate data can be as follows Figure 2 The spatial substrate data may be obtained from a data storage medium, such as an internal storage device of a computing device implementing method 100 or a database connected to the computing device via a communication interface. The stored spatial substrate data may be associated with a substrate ID to allow the spatial substrate data to be obtained using the substrate ID. The substrate ID may be input by a user via a graphical user interface, which allows the user to input the substrate ID or select the substrate ID from a list of substrate IDs available for spatial substrate data.

[0197] The coating material data includes data indicating the identity of the coating material. The data indicating the identity of the coating material may include the name of each coating material type, i.e., paint and varnish. The data indicating the identity of the coating material may include the ID of each coating material type. Providing the coating material data may include displaying a user interface by a computing device, thereby allowing a user to select a coating material type from a displayed list of available types. The user interface may allow a user to type in an ID or name of each coating material type. The user interface may allow a user to provide an ID / barcode / QR code of a coating material to be applied by a coating tool of the robot, and the computing device may obtain data indicating the identity of the coating material from a data storage medium based on the provided ID / barcode / QR code.

[0198] The coating material data may further include data indicating the identity of the coating material (such as an ID, barcode, QR code), property data of the coating material (such as chemical and / or physical property data), data about the composition of each coating material, or a combination thereof. The coating material data may further include data indicating the space substrate to be coated with (multiple) coating materials. If multiple space substrates are to be coated with different coating materials, this may be particularly preferred to ensure that the correct (multiple) coating materials are applied to each space substrate. The property data and the data about the composition can be stored on a data storage medium and can be obtained by a computing device using the provided data indicating the identity of the coating material.

[0199] The order in which the color paint and clear coat are applied using the robotic coating tool may not be provided by the user, but may be determined by the computing device using a set of rules stored on a data storage medium connected to the computing device. The order in which the color paint and clear coat are applied using the robotic coating tool may not be provided by displaying a user interface prompting the user to select the order in which the provided coating material types are provided.

[0200] The data indicating the coating tool may not be provided by the user, but may be obtained from a data storage medium based on the provided coating material type or coating material ID. The data indicating the coating tool may be provided by displaying a user interface prompting the user to select a coating tool (e.g., by displaying a list of available coating tools).

[0201] In box 104, method 100 can determine whether the surface of the space substrate includes at least one shielding material based on the space substrate data provided in box 102, which step is generally optional. Box 104 can be performed to reduce the consumption of the applied coating material and avoid overspray, because when determining the tool path data and thus determining the robot path data, the area covered with the shielding material can be excluded to avoid applying the coating material to the shielding material. Box 104 can be performed by a computing device that executes other boxes of method 100, or by another computing device (such as a server device that is connected to the computing device via a communication interface and can access a trained machine learning model for determining whether there is a shielding material on the space substrate). In this case, the computing device can act as a client device and can provide the space substrate data to the server device. Then, the server device can use the provided space substrate data and the trained machine learning model to determine whether there is a shielding material, and the determination result can be provided to the computing device. Using another server device to determine whether there is a shielding material may be beneficial because the trained machine learning model can be stored in a database that can only be accessed by the server, thereby reducing the overall system complexity. Additionally, the server device may use user feedback regarding the correct determination of occluding materials to improve the trained machine learning model using well-known learning techniques.

[0202] Determining whether the spatial substrate includes at least one shielding material based on the provided spatial substrate data may include generating a three-dimensional (3D) model using the provided spatial substrate data, and providing the generated 3D model to a trained machine learning model, in particular a deep learning algorithm. The 3D model may be created using 3D point cloud data and color data contained in the provided spatial substrate data, i.e., the 3D model is a 3D color model. Generating a 3D model based on point cloud data and color data may be performed according to methods well known in the prior art.

[0203] The machine learning model may have been trained on historical data of shielding materials, in particular on historical 3D color models of space substrates containing (multiple) shielding materials, to determine whether (multiple) shielding materials exist on the surface of the space substrate based on the provided historical space substrate data. The machine learning model, in particular the deep learning algorithm, can be hosted by a computing device, a remote server, or a cloud or other server implementing method 100. Advantageously, by locating the algorithm on a remote server or a cloud server, the cost of memory and / or a more complex processor added when using the algorithm to determine whether (multiple) shielding materials exist on the surface of each space substrate can be avoided. In addition, continuous or regular improvements to the algorithm can be more easily completed on a centralized server, and the data cost and risk of pushing algorithm updates to each computing device are avoided. The remote server can also be used as a central repository to store training data and / or operational data sets sent from various computing devices for training and developing existing algorithms. For example, a growing data repository can be used to update and improve algorithms on existing systems and provide improved algorithms for future use.

[0204] The trained machine learning algorithm, more specifically the artificial neural network (ANN) model used in block 104 can be obtained by training the ANN model to determine whether there is an obstructing material on the 3D color model of the spatial substrate using well-known machine learning methods. An exemplary commercially available software for implementing the training process is Keras (available on the Internet at Keras.io), which is an open source ANN model library that can run on TensorFlow or Theano, the latter providing the required computational engine. TENSORFLOW (an unregistered trademark of Google of Mountain View, California) is an open source software library originally developed by Google of Mountain View, California, and available as an Internet resource at www.tensorflow.org.

[0205] The model training dataset used to train a machine learning model can be divided into three parts: a training set, a validation set, and a verification (or "test") set. The training set is used to adjust the internal weighting algorithms and functions of the hidden layers of the neural network so that the neural network iteratively "learns" how to correctly identify and classify patterns in the input data. However, the verification set is mainly used to minimize overfitting. The verification set does not usually adjust the internal weighting algorithm of the neural network like the training set, but verifies whether any accuracy improvement relative to the training dataset will lead to an accuracy improvement relative to a dataset that has not been previously applied to the neural network or at least the network has not been trained on it (i.e., the verification dataset). If the accuracy relative to the training dataset is improved, but the accuracy relative to the verification dataset remains the same or decreases, the process is generally referred to as overfitting the neural network, and training should be stopped. Finally, the validation set is used to test the final solution in order to confirm the actual predictive ability of the neural network.

[0206] In one example, approximately 70% of the developed or collected data model set is used for model training, 15% is used for model verification, and 15% is used for model validation. These approximate divisions can be changed as needed to achieve the desired results. For example, approximately 300 sets of data can be collected, each set of data including 3D models of space substrates containing (multiple) shielding materials and without shielding materials. The training data set can include samples within the entire expected range of space substrates and the location and type of (multiple) shielding materials on the substrate.

[0207] The results of the determination may be provided to a computing device, and the computing device may display the results of the determination in a graphical user interface to allow a user to check whether the shielding material present on the spatial substrate is correctly identified. The results provided may represent an overlay of a 3D model processed by the machine learning model on an unprocessed 3D model (i.e., a 3D model generated by the spatial substrate data). A 3D color model including a classification of which surface areas contain (multiple) shielding materials may be provided to a display device for display on a screen (e.g., within a graphical user interface). This allows a user to check whether the trained machine learning model correctly detects the presence and location of (multiple) shielding materials, and if the (multiple) shielding materials present on the spatial substrate are not correctly determined, allows their presence and location to be corrected.

[0208] In box 106, a computing device implementing method 100 may obtain coating tool parameter data based on the coating material data provided in box 102. If more than one coating material is to be applied, that is, the coating material data provided in box 102 may contain data about more than one type of coating material, then coating tool parameter data associated with each type of coating material contained in the provided coating material data may be obtained in box 106. The coating tool parameter data may include coating tool tolerance data and at least one application parameter associated with a coating tool used by a robot, the coating tool being used to coat at least a portion of a surface of a space substrate with a corresponding coating material. The coating tool parameter data may be stored in a database connected to the computing device via a communication interface, and the computing device may use the data contained in the coating material data provided in box 102 (such as the name or ID of the type of coating material to be applied) to obtain appropriate coating tool parameter data. Obtaining coating tool parameter data based on the coating material data provided in box 102 may include obtaining data indicating the type of coating material (i.e., paint and varnish) contained in the provided coating material data, and obtaining coating tool parameter data associated with the paint and varnish.

[0209] The coating tool tolerance data may include target distance data, overlap percentage data, pattern size data, (multiple) rotation tolerances around the z-axis of the coating tool, (multiple) rotation tolerances around the x-axis of the coating tool, and (multiple) rotation tolerances around the y-axis of the coating tool. The target distance data and the overlap percentage may refer to a defined target distance value / overlap percentage and / or a target distance range / overlap percentage range. For example, for a paint material, the target distance data may include a range of 4 to 8 inches, and the overlap percentage data may include a range of 70% to 90%. For a varnish material, the target distance data may include a range of 6 to 10 inches, and the overlap percentage data may include a range of 40% to 60%. The target distance data and the overlap percentage data may each include a defined value (such as 6 inches for a paint material and 8 inches for a varnish material) and an allowable range that allows the generation of appropriate tool path data. The combination of a defined value and a range allows the use of the defined value as a starting point for generating tool path data, and if it is necessary to generate tool path data with the desired quality, the starting point may be adjusted according to the provided range. The overlap percentage data can remain fixed, while other parameters can be adjusted during the generation of tool path data. The overlap percentage data can be adjusted during the generation of tool path data. The pattern size data can be fixed for a specific target distance and can be scaled with the target distance. For example, if a pattern 10 inches high and 1.5 inches wide is assumed to be used for a target distance of 6 inches, and a pattern 14 inches high and 2.5 inches wide is assumed to be used for a target distance of 12 inches, then linear interpolation can be used between these two points to obtain the pattern size data between these two points. The pattern size data can include rules that associate a given target distance with a specific pattern size or pattern size range. The rotation tolerance around the z-axis (roll) and y-axis (yaw) of the coating tool can be within 15° of the normal surface vector, and the rotation tolerance around the x-axis (pitch) of the coating tool can be within 5° of the normal surface vector.

[0210] At least one application parameter associated with the coating tool may include the flow rate of the coating material, the voltage applied to the coating material, the pressure applied to the coating material, and the bell speed of the coating tool. This may be preferred if the coating tool corresponds to an ESTA (electrostatic spray application) applicator. At least one application parameter associated with the coating tool may include the flow rate of the coating material and the pressure applied to the coating material. This may be preferred if the coating tool corresponds to a pneumatic coating material applicator.

[0211] In box 108, the computing device implementing method 100 can obtain coating program data (CPD). The obtained coating program data may include a set of rules for coating outer edges and edges adjacent to (multiple) open spaces and a set of rules for coating the main surface. The set of rules for coating the outer edges of the space substrate and the edges adjacent to (multiple) open spaces may include at least one algorithm for determining the outer edges and at least one algorithm for determining the edges adjacent to (multiple) open spaces present in the surface space substrate. The algorithm for determining the edges adjacent to (multiple) open spaces may further allow for determining (multiple) open spaces present in the space substrate, which are at least partially surrounded by the determined edges adjacent to (multiple) open spaces. The set of rules for coating the outer edges and the edges adjacent to (multiple) open spaces may include rules for coating the edges adjacent to (multiple) open spaces, rules for coating the outer edges, and optional rules for coating (multiple) open spaces. At least the rules for coating edges adjacent to (multiple) open spaces and the rules for coating (multiple) open spaces may include coating tool tolerance data that is different from the coating tool tolerance data contained in the obtained coating tool parameter data. This allows the obtained coating tool tolerance data to be modified in the event that an edge adjacent to (multiple) open spaces and therefore also (multiple) open spaces are detected, and thereby allows the (multiple) open spaces and their edges and outer edges to be coated using different target distances and / or overlap percentages and / or pattern sizes and / or (multiple) rotation tolerances.

[0212] The rule set for coating the main surface may include the rule for determining the start of the coating program, the rule of the coating direction, the rule of the coating tool rotating in the tool path and the rule of surface separation. The rule set for coating the main surface may include more or less rules. The rule for determining the start of the coating program may, for example, include determining the corner of the substrate and continue in the direction of the vertical orientation of the coating material reservoir that allows the coating tool to be kept. The rule of the coating direction may, for example, include coating the substrate from top to bottom or from bottom to top and / or limit the curvature of the edge of the (multiple) tool path relative to the substrate, so that the (multiple) tool path does not follow the curvature or only follows the curvature to a certain extent. The rule for the coating tool to rotate in the tool path may include, for example, limiting rotation, so that the coating tool does not change orientation during the surface (or target area) of the coating space substrate. The rule of surface separation may include determining an independent surface and coating a separated (that is, adjacent) surface. The existence of an independent surface (or different target areas) can be determined by determining whether the space substrate includes a surface and a radius of curvature with a certain relative angle between the resulting two faces. For example, if the determined relative angle is from 25° to 90° and the radius of curvature is from 1 inch to 10 inches, the separation of the surfaces is given. Ensuring that the ranges of relative angles and radius of curvature are met can ensure that small body lines with small radii of curvature but little difference in angle between the two resulting faces are determined as independent surfaces, thereby avoiding unnecessary separation of spatial substrate surfaces. Coating independent surfaces can include coating adjacent surfaces in an adjacent-next pattern to maintain a wet film on two adjacent surfaces. This ensures that the newly coated surface is wet and can accept overspray, and that the adjacent surface to be coated still has wet overspray when coated.

[0213] The coating program data obtained may further include a rule set for applying at least two different coating materials on the same space substrate. If the coating process allows more than one coating material (such as a paint material followed by a varnish material) to be applied to the same space substrate, this may be preferred. For example, if the coating process only allows a single coating material to be applied to the same space substrate, the coating program data may not include the rule set. The rule set may include rules for copying coating tool tolerance data, which are used to apply previous coating materials (in this case, paint materials) and modify the copied data based on the coating material data provided in frame 102. In addition, the rule set may include rules for eliminating or executing the determination of the edges adjacent to (multiple) open spaces and therefore (multiple) open spaces. This allows (multiple) open spaces to be coated with another coating material (in this example, a varnish material), or avoids coating (multiple) open spaces with another coating material (for example, if a varnish layer is not required on (multiple) open spaces coated with a paint layer).

[0214] By using a rule set during the generation of robot path data, such robot path data can be reliably generated for substrates with large geometric variations. This allows such substrates to be coated by a robotic system using the generated robot path data in order to obtain a coating with high optical and mechanical quality, regardless of the geometry of the respective substrate. Thus, such a rule set can be used to customize the robot path data to the respective substrate without requiring manual interaction to ensure that the geometry of the substrate is fully considered during the generation of the robot path data, thereby avoiding inappropriate coating material application that may result in reduced optical and / or mechanical quality of the resulting coating.

[0215] In block 110, the computing device implementing method 100 may retrieve substrate type data (STD) based on the spatial substrate data provided in block 102. To this end, at least one processor included in the computing device may determine the substrate classification data included in the spatial substrate data provided in block 102, and may retrieve the associated substrate type data, for example, from a database or an internal memory of the computing device. The retrieved substrate type data may include at least one rule set, the at least one rule set including at least one rule for coating edges adjacent to (multiple) open spaces of a corresponding type of spatial substrate, at least one rule for coating (multiple) open spaces present within the spatial substrate, and at least one rotational tolerance of a coating tool. The at least one rule set may include more or fewer rules or may not include (multiple) rotational tolerances of a coating tool.

[0216] In block 112, if the optional block has been executed, the computing device implementing method 100 may generate tool path data based on the obtained coating tool parameter data, the obtained coating procedure data, the obtained substrate type data, and the determination result of block 104. The tool path data may be as follows: Figure 3A and Figure 3B Generate as described in .

[0217] In block 114, a computing device implementing method 100 may generate robot path data based on the tool path data generated in block 112. Generating the robot path(s) may include determining a collision geometry within the robot's workspace based on the spatial substrate data, and determining the robot path data based on the determined collision geometry and the generated tool path data. The determination of the collision geometry ensures that the robot does not impact and damage the spatial substrate(s) during the coating procedure. An initial pose of the robot may be determined, and based on the initial pose, all subsequent robot movements may be generated, thereby allowing the robot, particularly with respect to the coating process, to be positioned relative to the spatial substrate. Figure 7The movable robot member described follows the generated tool path data and returns to the initial pose. The generated tool path data may be sorted before the robot path data is generated. The generated robot path data may be as follows Figure 4 Optimize as described.

[0218] In block 116, the robot path data generated in block 114 may be provided via the communication interface. This may include providing the generated robot path data to a robot controller connected to the computing device and to the robot via the communication interface, as described later with respect to Figure 7 The generated robot path data may be further correlated with the ID of the spatial substrate and may be provided to a data storage medium (such as a database or internal memory). After the end of block 116, the method 100 may end, or may return to block 102 (for example, if it detects that spatial substrate data and coating material data are provided).

[0219] Figure 2 Demonstrates the use of Figure 1 102 of the block 102 of the method 200 for generating spatial substrate data. The generated spatial substrate data may include a substrate ID and a set of rules for smoothing the surface of a 3D model generated from data indicating the geometry and color of each spatial substrate. The method 200 may be performed by Figure 1 The described computing device is executed, the computing device including at least one processor, the at least one processor implementing a routine to perform the steps described with respect to the following blocks 202 to 216.

[0220] In box 202, the routine of implementing method 200 can detect user input indicating the substrate classification associated with each spatial substrate and user input indicating the position of each spatial substrate in the robot's workspace. User input can be detected by displaying a user interface on the display of the computing system implementing method 100, which allows the user to select the substrate classification of each spatial substrate present in the robot's workspace, such as by displaying a list of possible substrate classifications (such as trunk lids, hoods, fenders, etc.), or by displaying a text field and prompting the user to enter the corresponding (multiple) substrate classifications. The user interface can also allow the user to select the location of each spatial substrate in the robot's workspace (i.e., spray booth), such as by displaying a map or image of the spray booth and prompting the user to select the location of each spatial substrate in the displayed map of the spray booth. The spray booth can be divided into different zones, such as 5 zones, and the user can be prompted to select all zones including the spatial substrate. These zones can be marked in the spray booth to facilitate the user to select (multiple) zones. User input may be detected via an interactive element, such as an input device or an input / output device, in particular a mouse, keyboard, trackball, touch screen or a combination thereof, which may be attached to the computing device implementing the method 200 via a communication interface.

[0221] In frame 204, the position of each spatial substrate in the workspace of the robot and the substrate classification data can be determined based on the detected user input. If the user input is detected with a touch screen device, the processor located in the touch screen device can detect the user input and can determine the substrate classification data and the position of each spatial substrate based on the detected user input. Then, the determined data can be provided to the computing device of the implementation method 200. If the user input is detected via a mouse, keyboard or trackball attached to the computing device of the implementation method 200, the position of the substrate classification data and the substrate can be determined by the (multiple) processors contained in the computing device. Frames 202 and 204 can be executed at any time before frames 206 to 212, after frames 212 or 214, or before frame 216.

[0222] In block 206, data of the robot's workspace may be provided via a communication interface to a routine implementing method 200. The data of the robot's workspace may be provided by scanning the robot's workspace with a scanning device (or sensor system) attached to a movable robot member of the robot, as described below with respect to Figure 7 The data of the robot's workspace may include 3D point cloud data and color data (such as RGB or static image data). The computing device implementing method 200 may be used to control a robot including a scanning device via a robot controller, as described below with respect to Figure 7As described. For example, the computing system may allow a user to issue (multiple) commands to the robot controller via the computing device regarding scanning the robot workspace. The scanning of the robot workspace may also be automatically initiated, for example, by issuing corresponding commands to the robot controller by the computing device at the start of the method of the present invention. The robot controller may then control the robot including the scanning device to scan the workspace, and may provide data acquired by the scanning device to the computing device via the robot controller.

[0223] In block 208, the routine implementing method 200 may determine collision geometry present in the workspace based on the data of the workspace of the robot provided in block 206. The collision geometry may be generated by filtering the provided data of the workspace of the robot to reduce the number of data points; determining whether the data contains geometry having a certain size; creating (multiple) 3D objects present in the workspace from extreme points; and filling the generated (multiple) objects with a volume to obtain the collision geometry.

[0224] In box 210, the routine implementing method 200 can determine scanning path data of (multiple) scanning paths to be followed by the scanning device along the surface of each spatial substrate based on the determined position of each spatial substrate within the robot's workspace and the determined collision geometry, and the determined scanning path data can be provided to the scanning device via the communication interface. If the spatial substrates are present in more than one zone of the spray booth, or multiple spatial substrates are present in different zones, these zones can be traversed in a predefined order during the generation of the scanning path data to ensure that the scanning device scans the entire surface of each spatial substrate. The determined scanning path data can be provided to the scanning device (for example, regarding the scanning path data of the scanning device) via the communication interface. Figure 7 A sensor device 706 described above can then scan the surface of each spatial substrate based on the provided scanning path data. The scanning device can be the same scanning device used to acquire the data of the robot's workspace provided in box 206, or it can be a different scanning device. The scanning device can be attached to a movable robot member of the robot, and the determined scanning path data can be provided from a computing device implementing method 200 to a robot controller that controls the robot including the scanning device. Performing the scan provides detailed information about the spatial substrate and allows tool path data to be generated so that the resulting coating has a consistent high optical quality in appearance.

[0225] In box 214, the routine implementing method 200 can obtain data indicating the geometry and color of each spatial substrate obtained by the scanning device based on the scan path data determined and provided to the scanning device in box 212. For this purpose, the acquired scan path data can be assigned to a position ID, and the routine can associate the position ID with the determined position of each spatial substrate to assign the acquired scan path data to the corresponding spatial substrate. The acquired data may include 3D point cloud data and RGB color data or static image data. The acquired scan data can be stored by the scanning device or by a controller that controls the scanning device (such as a robot controller or another computing device), and the data can be obtained from the storage device. The acquired scan data can be provided to a computing device implementing method 200 in real time, and the computing device can store the received scan data.

[0226] In box 216, the routine implementing method 200 may generate spatial substrate data for each spatial substrate by combining the data obtained in box 214 with corresponding determined substrate classification data, the corresponding data indicating the identity of the spatial substrate and a corresponding rule set for smoothing the surface of the generated 3D model. The rule set may be obtained using the substrate classification data determined in box 204, such as by using a database containing rule sets, each rule set being associated with appropriate substrate classification data. The spatial substrate data for each spatial substrate may be generated by combining the data obtained in box 214 with corresponding determined substrate classification data. The spatial substrate data generated for each spatial substrate in box 214 may be stored on a data storage medium (such as an internal memory), and the data may then be used to determine the spatial substrate classification data. Figure 1 After the completion of block 216, method 200 may proceed to the previously described Figure 1 Frame 104.

[0227] Figure 3A and Figure 3B Demonstrates the use of Figure 1 The flowchart of the method 300 for tool path data is described in block 114 of FIG. The method 300 may be composed of Figure 1 The described computing device is executed, the computing device including at least one processor, the at least one processor implementing a routine to perform the steps described with respect to the following blocks 302 to 326.

[0228] In block 302, the routine for implementing method 300 may be based on Figure 1 The spatial substrate data provided in block 102 of the present invention is used to generate a three-dimensional (3D) model of each spatial substrate. The provided spatial substrate data may be as described above with respect to Figure 2The spatial substrate data may be generated as described above. For each spatial substrate, the spatial substrate data may include 3D point cloud data, static image data of the spatial substrate, and a set of rules for smoothing the surface of the generated 3D model. The generation of a 3D color model from the 3D point cloud data and static image data may be performed using known methods and software such as the open source library Open3D.

[0229] In block 304, the routine implementing method 300 may retrieve a corresponding rule set and may apply the retrieved rule set to smooth the surface of the 3D model generated for each spatial substrate in block 302, which step is generally optional. The rule set may be included in Figure 1 In the spatial substrate data provided in box 102 of . The routine of implementing method 300 can obtain a rule set from a data storage medium based on the substrate classification data contained in the provided spatial substrate data, and the data storage medium includes a rule set that is interrelated with the substrate classification data. Smoothing of each generated 3D model can involve determining the surface (or area) of the 3D model that matches the surface data contained in the (multiple) rules of the rule set, and applying an optimization algorithm (such as a numerical smoothing algorithm known in the prior art) on the determined surface in combination with the threshold value contained in the (multiple) rules. The threshold value can be used to control the degree of smoothing performed by the optimization algorithm. For example, if the spatial substrate type is a hatch, the corners will be smoothed to a greater extent and the cavities of the inner edges and taillights will be determined, while on the front side panels, the corners will remain sharper and any holes that may be regarded as inner edges will be ignored. Using a rule set to smooth (multiple) 3D models allows control of the degree of smoothing, and therefore the resulting quality of the coating in terms of appearance, because the higher the degree of smoothing, the lower the resulting quality. A higher degree of smoothing may be used for surfaces that do not require high quality, such as headlight recesses present in a fender, while a lower degree of smoothing may be used for surfaces that require higher quality, such as portions of a fender that are visible after the coated fender is mounted to a vehicle body. Smoothing the 3D model(s) allows for more efficient generation of tool path data because the meshes of the 3D model(s) obtained from the point cloud data are converted into surfaces that may be more efficiently processed during generation of the tool path data, for example by a computer processor(s) generating the tool path data, as described later.

[0230] In block 306, the routine implementing method 300 may be based on Figure 1The coating process data obtained in the box 108 of the routine and each 3D model generated in the box 302 or each 3D model smoothed in the box 304 are used to determine (multiple) outer edges, (multiple) edges adjacent to (multiple) open spaces, major surfaces, and (multiple) open spaces present in each space substrate. For this purpose, the routine can obtain an appropriate set of rules and apply the rule set to each generated / smoothed 3D model. The routine can obtain an algorithm from the rule set and can apply the algorithm to each smoothed 3D model to determine the outer edge, (multiple) edges adjacent to (multiple) open spaces, major surfaces, and (multiple) open spaces, as described above.

[0231] In block 308, the routine implementing method 300 may be based on Figure 1 The coating tool parameter data and coating program data associated with the first coating material (in this example, the paint material) obtained in the boxes 106 and 108 of Figure 1 The tool path data for the outer edge and the tool path data for the edge adjacent to the open space(s) are generated by using the substrate type data obtained in block 110 of the embodiment and each 3D model generated in block 302 or each smoothed 3D model generated in block 304. The tool path data for the outer edge and the tool path data for the edge adjacent to the open space(s) may be generated by:

[0232] - determining target distance(s), overlap percentage(s), pattern size, and rotational tolerance(s) of the coating tool based on the outer edges, edges adjacent to the open space(s) and the open space(s) determined in block 306, the derived coating tool parameter data, the derived coating procedure data, the derived spatial substrate data, and each generated and smoothed 3D model, and

[0233] - Using each generated and smoothed 3D model, tool path data is generated according to the determined target distance(s), overlap percentage(s), pattern size, and rotational tolerance(s) of the coating tool.

[0234] For example, if the spatial substrate is a fender or hood, the (multiple) open spaces present in the fender or hood (such as the marker light recess in the fender or the hood air intake recess in the hood) cannot be processed according to the (multiple) rules for coating the (multiple) open spaces contained in the substrate type data. The headlight recess, the top inner rail and the tab, and the lower flange on the dog leg can be determined as (multiple) open spaces to be coated according to the rules contained in the coating program data. If the spatial substrate is a vehicle door, the (multiple) open spaces present in the vehicle door (except windows), such as the door handle mounting holes, cannot be processed according to the (multiple) rules for coating the (multiple) open spaces contained in the substrate type data. The door frame and window frame can be determined as (multiple) open spaces to be coated according to the rules contained in the coating program data.

[0235] In block 310, the routine implementing method 300 may be based on Figure 1 The tool path data for the main surface of each spatial substrate is generated by using the coating program data associated with the first coating material (in this example, the paint material) obtained in the box 108, in particular the rule set for coating the main surface, and each 3D model generated in the box 302 or each smoothed 3D model generated in the box 304. The tool path data for the main surface can be generated in the following way

[0236] - determining, based on the acquired coating process data and each generated and smoothed 3D model, whether the main surface of each spatial substrate comprises at least two independent surfaces,

[0237] - determining whether a major surface of each spatial substrate comprises a convex shape, and creating a shell around the spatial substrate(s) using a convex hull method,

[0238] - determining target distance(s), overlap percentage(s), pattern size and rotational tolerance(s) of the coating tool based on the determined independent surfaces, the determined convex shape, the derived coating tool parameter data, the derived coating procedure data and each generated and smoothed 3D model, and

[0239] - Using each generated and smoothed 3D model, tool path data is generated according to the determined target distance(s), overlap percentage(s), pattern size, rotational tolerance(s) of the coating tool.

[0240] The rule set for coating the main surface can include the rule for determining the start of the coating program, the rule of coating direction, the rule of coating tool rotating in the tool path and the rule of surface separation. The rule set for coating the main surface can include more or less rules. The rule for determining the start of the coating program can, for example, include determining the corner of the substrate and continue in the direction of the vertical orientation of the coating material reservoir that allows the coating applicator to be kept attached. The rule of coating direction can, for example, include coating the space substrate from top to bottom or from bottom to top (such as a door), or from the leading edge to the trailing edge or from one side to the other side (such as a hood), and / or limit the curvature of (multiple) tool paths relative to the edge of the substrate so that (multiple) tool paths do not follow the curvature or only follow the curvature to a certain extent. The rule for coating tool rotating in the tool path can include, for example, limiting rotation so that the coating tool does not change orientation during the surface (or target area) of the coating space substrate. The rule of surface separation can include determining an independent surface and coating a separated (that is, adjacent) surface. In this example, the existence of independent surfaces (or different target areas) is determined by determining whether the space substrate includes a surface with a certain relative angle and a radius of curvature between the two resulting faces. For example, if the determined relative angle is from 25° to 90° and the radius of curvature is from 1 inch to 10 inches, the separation of the surfaces is given. Ensuring that the ranges of relative angles and radii of curvature are met can ensure that small body lines with small radii of curvature but little difference in angle between the two resulting faces are determined as independent surfaces, thereby avoiding unnecessary separation of the space substrate surface. Coating independent surfaces can include coating adjacent surfaces in an adjacent next mode to maintain a wet film on two adjacent surfaces. This ensures that the newly coated surface is wet and can accept overspray, and that the adjacent surface to be coated still has wet overspray when coated.

[0241] The determination of whether a spatial substrate includes a convex hull can be performed based on each generated or smoothed 3D model by determining the surface curvature of each spatial substrate from side to side. For example, such a convex surface may exist on a bumper cover or hood. A shell generated by applying a convex hull algorithm known in the prior art to the convex surface of the (multiple) spatial substrates can be used to generate tool path data for the convex surface.

[0242] The steps of determining whether the major surface of each spatial substrate includes at least two independent surfaces and / or convex surfaces can be performed in any order.

[0243] In box 312, the routine implementing method 300 can use the tool path data generated in boxes 308 and 310 to calculate the coating material application for each spatial substrate. Using the generated tool path data to calculate the corresponding coating material application can include using the generated tool path data to simulate the coating material application. The simulation allows the calculation of the coverage map of each generated or smoothed 3D model. The calculated (multiple) coverage maps can be provided to a display device via a communication interface, for example, which displays the received (multiple) coverage maps in a graphical user interface (GUI). The (multiple) coverage maps can include different colors to indicate whether the (multiple) predefined parameters are met. The (multiple) coverage maps can also include the calculated coating material application and the (multiple) predefined parameters obtained to allow visual comparison.

[0244] In box 314, the routine implementing method 300 can determine whether each coating of each space substrate generated by the calculated coating material application on each space substrate meets at least one predefined parameter. To this end, the routine can compare the calculated coating material application of each space substrate with the predefined parameters of each space substrate obtained, and determine whether the calculated coating material application is within or outside the predefined parameters obtained. The routine can use the provided material coating data (e.g., the ID of the coating material or the type of coating material) to obtain the predefined parameters from the database. The database may contain the (multiple) parameters associated with the coating material ID or the coating material type. The dry film and / or wet film thickness or its range and / or the surface area to be coated or its range can be obtained as (multiple) predefined parameters. The (multiple) coverage maps calculated in box 312 can be used to determine whether a specific target area of ​​the 3D model of each space substrate will be coated with an appropriate percentage of coating material so that the wet film and / or dry film thickness of the resulting coating meets the predefined range of the wet film and / or dry film thickness obtained. The overlay(s) may also be used to control the thickness of clear coating materials, or to control the coating thickness on portions of spatial substrate(s) that require specific specifications for transmittance, such as ADAS (Advanced Driver Assistance Systems) and radar sensors.

[0245] If at least one predefined parameter is not met, the routine may return to block 308, and blocks 308 and 310 may be repeated using the determination result of block 314. After repeating blocks 308 and 310 to generate new tool path data, the newly generated tool path data may be checked using the steps described above in blocks 312 and 314. The loop may be repeated until it is determined in block 314 that the generated tool path data meets at least one, particularly all, predefined parameters. Executing blocks 312 to 314 ensures that the coating produced by applying the coating material using the robot path data generated from the tool path data meets predefined quality parameters, such as wet and / or dry film thickness and the surface area to be coated. If the routine implementing method 300 determines in block 314 that at least one, particularly all, obtained predefined parameters are met, it may proceed to block 316.

[0246] In block 316, a routine implementing method 300 may determine whether additional coating materials are to be applied in addition to the first coating material. This determination may be based on Figure 1 The routine may proceed to block 318 based on the data contained in the coating material data provided in block 102 of FIG. 1 . For example, if the data contains data for at least two different coating materials, the routine may use the information to determine the amount and type of coating material to be applied. If the routine determines that additional coating materials are to be applied, it may proceed to block 318. Otherwise, the routine may proceed to the previously described Figure 1 Frame 116.

[0247] In block 318 , a routine implementing method 300 may generate tool path data based on coating program data and coating tool parameter data associated with the additional coating material as described with respect to blocks 308 and 310 .

[0248] In block 320 , a routine implementing method 300 may use the tool path data generated in block 318 to calculate coating material application, as described with respect to block 312 .

[0249] In block 322, the routine implementing method 300 may determine whether the coating material application calculated in block 320 satisfies at least one, and in particular all, predefined parameters, as described with respect to block 316. If at least one predefined parameter is not satisfied, the routine implementing method 300 may return to block 318 and may optimize the generated tool path data, as described with respect to block 314. If at least one predefined parameter is satisfied, the routine may proceed to block 324.

[0250] In block 324, the routine implementing method 300 may determine whether at least one additional coating material is to be applied, as described with respect to block 316. If the routine determines that at least one additional coating material is to be applied, it may proceed to block 318 described above, otherwise it may proceed to Figure 1 Frame 116.

[0251] Figure 4 Demonstrates the use of Figure 1 Flowchart of method 400 for robot path data as described in block 114 of the present invention. Method 400 may be comprised of Figure 1 The described computing device is executed, the computing device includes at least one processor, and the at least one processor implements a routine to perform the steps described with respect to the following blocks 402 to 410.

[0252] In box 402, a routine implementing method 400 may determine whether to sort the generated tool path data. The determination may be made based on data contained in the obtained coating program data or based on the programming of the routine. For example, the obtained coating program data may contain rules that determine that tool path data generated for outer edges and edges adjacent to (multiple) open spaces may be executed before tool path data generated for (multiple) open spaces and (multiple) main surfaces. Executing the (multiple) tool paths for outer edges and edges adjacent to (multiple) open spaces before the (multiple) tool paths for (multiple) open spaces and main surfaces may be beneficial because coating of the edges may produce overspray on the main surfaces, which may then be covered by coating the main surfaces, thereby avoiding a negative impact on the final overall appearance. In addition, this avoids applying too much coating material to the edges, which is disadvantageous because the presence of too much coating material on the edges is prone to sagging, sag, and heavy edges, thereby negatively affecting the final overall appearance of the coated substrate. If the routine determines in box 402 that the tool path data to be sorted is to be sorted, the tool path data may be sorted. Figure 1 If the tool path data generated in block 112 is sorted, it may proceed to block 404, otherwise it may proceed to block 406, as described later.

[0253] In block 404, the routine implementing method 400 may perform a Figure 1The tool path data generated in the box 112 of the method are sorted. The obtained rules may be contained in the coating program data, or may be obtained by the routine from a data storage medium (such as a database) based on the provided coating material data. The generated tool path data may be sorted according to the (multiple) rules contained in the obtained coating program data and according to the rules obtained based on the provided coating material data. For this purpose, the routine may obtain appropriate rules from the obtained coating program data and may apply these rules on the generated tool path data to sort the tool path data accordingly. The generated tool path data may include metadata indicating whether tool path data has been generated for an outer edge, an edge adjacent to (multiple) open spaces, (multiple) open spaces, or a main surface, so as to use the metadata to sort the generated tool path data. If more than one space substrate is to be coated, the routine may sort the generated tool path data so that all tool paths associated with a specific coating material type (such as a paint material) are executed before all tool paths associated with another coating material (such as a varnish material). However, the tool path data may also be sorted so that one space substrate is completely coated before coating another space substrate. The tool path associated with a specific coating material (such as a primer coating material) can be executed before the tool path associated with the color paint material and the clear coat material for each specific spatial substrate, that is, the tool paths for the color paint material and the clear coat material or each spatial substrate can be grouped and executed in sequence so that the color paint material and the clear coat material are applied to each spatial substrate after the primer coating material is applied to all spatial substrates.

[0254] In block 406, the routine implementing method 400 may use the tool path data generated in block 112 to generate robot path data, such as with respect to Figure 1 as described in block 114 of FIG.

[0255] In block 408, the routine implementing method 400 may determine whether to optimize the generated robot path data. This determination may be made according to the programming of the routine. Optimizing the generated or sequenced robot path data may be beneficial because it allows the robot's motion to be smoothed to make it more efficient and consistent. If the routine determines in block 408 that the generated robot path data is to be optimized, it may proceed to block 410. Otherwise, as described above, it may proceed to Figure 1 Frame 116.

[0256] In block 410, the routine implementing method 400 may optimize the generated robot path data. The optimization of the generated robot path data may be performed, for example, using available open source libraries such as Descartes (ROS-Industrial project for performing path planning on undefined Cartesian trajectories) and software frameworks such as trajopt (a software framework for generating robot trajectories through local optimization). After block 410 ends, the routine may proceed to Figure 1 Frame 116.

[0257] Figure 5 A flow chart of a second non-limiting embodiment of a method 500 for generating robot path data for a robot including a coating tool to follow during coating of at least a portion of a surface of a space substrate with at least one coating material is presented. The method 500 may be used to coat at least a portion of a surface of a space substrate with a coating material using a robotic system including a robot including a coating tool. The method 500 may include as described above with respect to Figure 1 The method 500 may be performed by referring to the blocks 102 to 116 described above and further blocks 502 to 506 described below. Figure 1 The method 500 may be performed by a server device connected to a server device for execution via a communication interface. Figure 1 A computing device that is at least a portion of the frame of the computer. In this arrangement, the computing device can act as a client device and can provide the generated tool path data and the provided coating material data to the server device. The server device can then perform calculations using the received data and can provide the results of the calculations to the client device. This arrangement can be performed if the computing power of the computing device is insufficient to perform the calculations.

[0258] In box 502, the routine implementing method 500 can determine whether to calculate the amount of each coating material required to coat at least a portion of the surface of each spatial substrate. This determination can be made according to the programming of the routine, or, for example, can be made by displaying a graphical user interface that prompts the user to select whether to perform the calculation, or by providing a corresponding button / menu item on the graphical user interface, which triggers the calculation when user input indicating selection of the button / menu item is detected. If the routine determines that the amount of each coating material is to be calculated, it can proceed to box 504. Otherwise, it can end method 500 or can proceed to Figure 1 Frame 102.

[0259] In block 504, the routine implementing method 500 may be based on Figure 1 The tool path data generated in block 112 and in Figure 1 The coating material data provided in block 102 is used to calculate the amount of each coating material required to coat at least a portion of the surface of each spatial substrate.

[0260] In block 506, the routine implementing method 500 may provide a calculated amount of each coating material for each spatial substrate. The calculated amount may be provided to a display device for display within a graphical user interface so that a user may prepare the desired amount. The calculated amount along with the provided coating material data may be provided to an automatic mixer via a communication interface, which uses the received data to automatically prepare the calculated amount.

[0261] Execution blocks 504 and 506 can allow a user to obtain information about the amount of coating material needed for the coating process and thereby allow the use of this data for inventory planning or determining the amount of coating material that needs to be prepared for the coating program. The latter avoids preparing more coating materials than required for the coating program, thereby allowing the cost associated with waste and waste coating materials to be reduced. In addition, the information (such as coating material ID) of the coating material contained in the determined amount and the coating material data can be provided to an automatic mixer, which then mixes the determined amount based on the received data. This allows the coating process to be fully automated, and avoids the waste coating material prepared due to mixing errors.

[0262] Can be Figure 1 Execute blocks 502 to 506 after block 116 of Figure 1 Blocks 502 to 506 are performed after block 114 and before block 116. This allows for the provision of generated robot path data and calculated amounts of each coating material.

[0263] Figure 6 A computing device 600 is shown that can be used to implement any of the methods described in the above figures. For example, Figure 6 The type of computing device 600 shown in FIG. 6 may be used to implement Figures 1 to 5 In all cases, computing device 600 represents a physical and tangible processing mechanism.

[0264] The computing device 600 may include one or more hardware processors 602. The hardware processor(s) may include, but are not limited to, one or more central processing units (CPUs), and / or one or more graphics processing units (GPUs), and / or one or more application specific integrated circuits (ASICs), etc. More generally, any hardware processor may correspond to a general purpose processing unit or a special purpose processor unit.

[0265] The computing device 600 may also include a computer-readable storage medium 604 corresponding to one or more computer-readable medium hardware units. The computer-readable storage medium 604 retains any kind of information 606, such as computer-readable or machine-readable instructions, settings, data, etc. Computer-executable instructions include, for example, instructions and data that, when executed at the processor 602, cause the computing device 600 to perform a specific function or group of functions. Alternatively or in addition, the computer-executable instructions may configure the computing device 600 to perform a specific function or group of functions. The computer-executable instructions may be, for example, binary or even instructions that undergo some conversion (such as compilation) before being directly executed by the processor, such as intermediate format instructions, such as assembly language, or even source code. Without limitation, for example, the computer-readable storage medium 604 may include one or more solid-state devices, one or more magnetic hard disks, one or more optical disks, tapes, etc. Any instance of the computer-readable storage medium 604 may use any technology for storing and retrieving information. Further, any instance of the computer-readable storage medium 604 may represent a fixed or removable component of the computing device 600. Further, any instance of computer-readable storage media 604 may provide for volatile or non-volatile information retention.

[0266] The computing device 600 may utilize any instance of the computer-readable storage medium 604 in different ways. For example, any instance of the computer-readable storage medium 604 may represent a hardware memory unit (such as random access memory (RAM)) for storing transient information during execution of a program by the computing device 600 and / or a hardware storage unit (such as a hard disk) for retaining / archiving information on a more permanent basis. In the latter case, the computing device 600 also includes one or more drive mechanisms 608 (such as a hard disk drive mechanism) for storing and retrieving information from the instance of the computer-readable storage medium 604.

[0267] When the hardware processor(s) 602 execute computer-readable instructions stored in any instance of the computer-readable storage medium 604, the computing device 600 may perform any of the functions described above. For example, the computing device 600 may execute the computer-readable instructions to perform the Figures 1 to 5 Each box of the method described in .

[0268] Alternatively or in addition, the computing device 600 may rely on one or more other hardware logic components 610 to perform operations using a task-specific set of logic gates. For example, the hardware logic component(s) 610 may include a fixed configuration of hardware logic gates that are created and set, for example, at the time of manufacture and cannot be changed thereafter. Alternatively or in addition, the other hardware logic component(s) 610 may include a set of programmable hardware logic gates that can be set to perform different dedicated tasks. The latter class of devices includes, but is not limited to, programmable array logic devices (PALs), general purpose array logic devices (GALs), complex programmable logic devices (CPLDs), field programmable gate arrays (FPGAs), and the like.

[0269] Figure 6 The hardware logic circuitry 612 is generally indicated to include any combination of hardware processor(s) 602, computer-readable storage media 604, and / or other hardware logic component(s) 610. That is, the computing device 600 may employ any combination of hardware processor(s) 602 that execute machine-readable instructions provided in the computer-readable storage media 604 and / or one or more other hardware logic components 610 that perform operations using a collection of fixed and / or programmable hardware logic gates. More generally, the hardware logic circuitry 612 corresponds to one or more hardware logic components of any type(s) that perform operations based on logic stored and / or otherwise embodied in the hardware logic component(s).

[0270] In some cases (e.g., where computing device 600 represents a user computing device), computing device 600 also includes an input / output interface 614 for receiving various inputs (via input devices 616) and for providing various outputs (via output devices 618). Illustrative input devices include a keyboard device, a mouse input device, a touch screen input device, a digitizer tablet, one or more still image cameras, one or more video cameras, one or more depth camera systems, one or more microphones, a voice recognition mechanism, any movement detection mechanism (e.g., an accelerometer, a gyroscope, etc.), etc. A particular output mechanism may include a display device 618 and an associated graphical user interface presentation (GUI) 620. Display device 618 may correspond to a liquid crystal display device, a light emitting diode display (LED) device, a cathode ray tube device, a projection mechanism, etc. Other output devices include a printer, one or more speakers, a tactile output mechanism, an archiving mechanism (for storing output information), etc. The computing device 600 may also include a communication channel 624 for communicating with other devices (such as Figure 7One or more network interfaces 622 for exchanging data with the robot controller 704 and databases 708, 710, 712, and 722. One or more communication buses 628 communicatively couple the above components together.

[0271] The communication conduits 624 may be implemented in any manner, for example, via a local area computer network, a wide area computer network (e.g., the Internet), a point-to-point connection, etc., or any combination thereof. The communication conduits 624 may include any combination of hardwired links, wireless links, routers, gateway functions, name servers, etc. governed by any protocol or combination of protocols.

[0272] Figure 6 Computing device 600 is shown as being composed of a discrete collection of individual units. In some cases, the collection of units may correspond to discrete hardware units provided in a computing device chassis having any form factor.

[0273] Figure 7 A block diagram of a robotic system 700 for coating at least a portion of a surface of a space substrate with at least one coating material according to an embodiment of the present invention is shown. The robotic system can be used to implement the method of the present invention (e.g., regarding the method of coating at least a portion of a surface of a space substrate with at least one coating material) by using a computing device 718 to generate robotic path data and providing the generated robotic path data to a robotic device 724 via a robotic controller 704. Figure 1 Method 100 described above).

[0274] The robotic system 700 may include a computing device 710 and a robotic device 702. The computing device 710 may be, for example, Figure 6 The computing device 600 described in the above and may include a CPU 712 and a storage medium 714. The CPU may be as described above with respect to Figure 6 The storage medium 714 may be a hardware processor or other hardware logic component as described above. Figure 6 Any computer-readable storage medium described herein. The computer-readable storage medium may store computer-readable instructions (such as program codes) that allow the computing device 710, in particular the CPU 712, to perform the method of the present invention (such as the method of Figure 1 Method 100 described and related Figure 5 Method 500 described).

[0275] The computing device 710 can be connected to a display device 726 via a communication interface, and the display device can receive data (such as in the form of a video or video) from the computing device. Figure 3A The 3D model generated in block 302 or 304, the tool path data generated, Figure 3A Box 312 and / or Figure 3BThe coverage map calculated in block 324 and / or in Figure 5 The amount of coating material calculated in block 504 of FIG. 5 ), and the received data may be displayed using a graphical user interface. To this end, the display device 726 or the computing device 710 may generate a user interface presentation containing the corresponding data, and may provide / display the generated user interface presentation. The display device 726 may be, for example, Figure 7 A mobile display device or a fixed display device (such as a computer monitor) is shown. The display device may include a screen (such as an LCD screen) to display a graphical user interface including data generated by the computing device 710.

[0276] The computing device 710 may be connected to different databases 716, 718, 720, 722, and 724 via a communication interface. Database 716 may contain spatial substrate data, database 718 may contain coating material data, database 720 may contain coating parameter data, database 722 may contain coating procedure data, and database 724 may contain substrate type data as previously described. The data contained in the databases may be associated with data that allows the computing device 710 to obtain the data. A portion of the data stored in the databases (such as the spatial substrate data) may be stored in the storage medium 714 instead of in the database. At least a portion of the data may be stored in the same database to reduce the number of databases connected to the computing device.

[0277] The computing device 710 may be connected to another computing device, such as a server device, that is different from the computing device 710 (not shown). The server device may be used, for example, to determine whether there is a shielding material on the space substrate (e.g., regarding Figure 1 ), or for performing the method of the present invention (such as with respect to Figure 1 Other blocks of the described method 100).

[0278] The system may further include a robotic device 702 including a robot controller 704 and a robot arm (or movable robot member) 706. The robotic device 702 may be connected to a computing device 710 via the robot controller 704 using a communication interface. The robot controller 704 may be configured to receive robot path data and optionally additional commands (such as commands for tool change) and to control (i.e., move) the robot using the received robot path data and additional commands. The robot controller may be located outside the spray booth to avoid negative impacts on the robot controller during spray operations (see Fig. 9). The robotic arm 706 may be any industrial robot with 'n' degrees of freedom to which the sensor system or coating tool 708 may be automatically attached. The robotic arm 706 may be located inside a spray booth (see Fig. 9 ). The robot arm may include a sensor system or coating tool 708. The robot arm 706 may not include a sensor system or coating tool 708. In this case, the system or coating tool 708 may be stored in a tool rack (see Fig. 12B ). The robotic arm 706 may include a tool changer that may be configured to attach different coating tools or sensor systems to the robotic arm 706. By providing the robot controller 704 with the location of the tool in the tool holder, the robotic arm 706 may pick up the appropriate coating tool or sensor system 708.

[0279] The sensor system 708 can be configured to generate data about the robot's workspace and to acquire data about the geometry and color of the space substrate based on the scan path data provided to the robot controller 704. Figure 2 The scan path data may be determined as described in block 210 of . The sensor system may include a depth sensor (such as a laser scanner).

[0280] Coating tool 708 may include a coating applicator, such as the one described later. Fig. 8A and Figure 8B The coating tool may further include a coating material reservoir, the reservoir comprising Fig. 12A and Fig. 12B Specific coating materials shown.

[0281] Fig. 8A 802 of a coating tool according to an embodiment of the present invention is shown. The spray applicator 802 may be an electrostatic spray applicator including a nozzle and a rotating bell 804. Fig. 8A As shown, the target distance can be the distance between the nozzle end of the spray applicator 802 and the surface of the space substrate 806.

[0282] Figure 8B Shown Fig. 8A The x-axis and y-axis of the spray applicator 802 of the coating tool. Figure 8B Shows Fig. 8A FIG. 8 is a top view of a spray applicator 802 and illustrates movement of the spray applicator 802 in vertical and horizontal directions.

[0283] Fig. 9 A spray booth 906 and a robotic system of the present invention (e.g. Figure 7Schematic diagram of an example system 900 of a robotic system 700).

[0284] The system 900 may include an electrical cabinet 902 that contains the power required to use the system 900. The system 900 may further include a cooler 904 and a heat exchanger 916 for providing power to the robotic device (e.g., Figure 7 The robotic arm 706 and / or coating tool 708 described herein purifies the air.

[0285] The system 900 may further include a spray booth 906 including a spray booth door 908 to allow the space substrate to be placed in the spray booth and to allow the coated space substrate to be removed from the spray booth. The spray booth may include a device for heating (not shown) so that the (multiple) coating materials applied by the robotic arm can be dried and / or cured in the spray booth without having to remove the coated space substrate and place the coated space substrate in a separate oven.

[0286] The system 900 may further include spray booth ventilation 910 to allow ventilation of the spray booth to remove residues of coating material or volatile compounds that evaporate from the applied coating material(s) while drying and / or curing the resulting coating film.

[0287] The system 900 can further include a robotic controller 912 external to the spray booth 906. This ensures that the robotic controller 912 is not negatively affected by the spray and volatile compounds present in the spray booth during spraying and curing. The robotic controller 912 can be connected to a robot 914. The robot 914 can include a robotic arm (see Fig.11 ). Suitable robots include robots configured for spray painting tasks, such as the FANUC P-50iB / 15, which is a 6-axis industrial robot commercially available from Fanuc. The robot can be secured to a Gail track system to allow the robot to move within the spray chamber.

[0288] Fig.10 Shown Fig. 9 906 and a top view of a spray booth 906 and a robot controller 912. The robot 914 can be fixed on a rod 1002 that allows the robot to move along the length of the spray booth 906 using a Gale track system so that the robot can reach possible locations of spatial substrates within the spray booth 906. The spray booth can further include an air outlet 1004 that allows air provided from the ceiling into the spray booth to flow out through the air outlet. The gantry axis motor configured to move the robot along the spray booth can reside in an explosion proof box 1006 to avoid explosions during spraying of solvent-based liquid coating materials containing explosive liquids.

[0289] Fig.11 Shown Fig. 9 906. The sensor system 706 may be attached to the robot 914. The robot 914 may not have an attached sensor system or coating tool. The robot 914 may have an attached coating tool (see Fig. 12A , Fig. 12B ). As previously described, the Gale track system 1102 can be used to move the robot 914.

[0290] Fig. 12A Shown Fig. 9 906 , which includes a tool rack containing multiple coating tools, a housing containing a sensor system, and a Gale track system 1102 .

[0291] The tool rack 1202 may include a plurality of coating tools 706. Each coating tool may include a spray applicator 802 and a coating material reservoir 1206 containing a particular coating material (see Fig. 12B ). Each coating tool can be automatically attached to the robot 914 using a tool changer (not shown).

[0292] The sensor device 706 may be housed within the housing 1204 to avoid contamination of the sensor device during a spraying process that produces a spray that may contaminate the sensor device 708. The robot 914 may be configured to open the housing and remove the sensor device from the housing 1204 using a tool changer.

[0293] Fig. 12B Shown Fig. 12A 8 and 1206. The tool rack shown in FIG. 8 and the enlarged view of the housing accommodating the sensor system. The tool rack can include multiple coating tools, such as 4 coating tools. Each coating tool can include the spray applicator 802 and the coating material reservoir 1206 described above about FIG. 8. The coating material reservoir can include specific coating materials, which can be prepared, for example, by mixing different materials (such as pigment paste, basic varnish, thinner, hardener) in the reservoir or by mixing the materials in a mixing device and filling the resulting coating material in the reservoir. Robot 914 can pick up corresponding coating tools based on the instructions provided by robot controller 912.

[0294] The present disclosure is also described in conjunction with preferred embodiments as examples. However, other variations may be understood and implemented by those skilled in the art and those practicing the claimed invention through a study of the drawings, the present disclosure, and the claims. It is worth noting that it is not required that different steps be performed somewhere or at a certain node in the distributed system, i.e., each step may be performed at different nodes using different devices / data processing units.

[0295] As used herein, "determining" also includes "initiating or causing determination", "generating", "querying", "accessing", "associating", "matching", "selecting" also includes "initiating or causing generation, accessing, querying, associating, selecting and / or matching", and "providing" also includes "initiating or causing determination, generation, accessing, querying, associating, selecting and / or matching, sending and / or receiving". "Initiating or causing performance of an action" includes any processing signal that triggers a computing processor to perform a corresponding action.

[0296] In the claims and the description, the word "comprising" or "including" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. A single element or other unit may fulfill the functions of several entities or items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to an advantageous embodiment.

Claims

1. A computer-implemented method for generating robot path data for a robot including a coating tool to follow during coating at least a portion of a surface of at least one spatial substrate with at least one coating material, the method comprising the following steps: (a) providing to at least one computer processor via a communication interface - spatial substrate data for each spatial substrate, the spatial substrate data comprising substrate classification data and data indicative of the geometry and color of each spatial substrate, and - coating material data, the coating material data comprising data indicating the type of the at least one coating material and optionally indicating the sequence of coating materials to be applied to the spatial substrate and / or data indicating the coating tool; (b) optionally determining, using the at least one computer processor, whether each spatial substrate includes at least one shielding material based on the provided spatial substrate data; (c) using the at least one computer processor to obtain via the communication interface - coating tool parameter data based on the provided coating material data, said coating tool parameter data comprising coating tool tolerance data and at least one application parameter associated with the coating tool, - coating program data comprising a set of rules for coating outer edges and edges adjacent to the open space(s) and a set of rules for coating the main surface, and - substrate type data based on the provided spatial substrate data, the substrate type data comprising a set of rules for spatial substrate types to match the substrate classification data; (d) generating, using the at least one computer processor, tool path data for the tool path(s) to be followed by the coating tool along the surface of each spatial substrate based on the data obtained in step (c) and optionally the results of the determination performed in step (b); (e) generating, with the at least one computer processor, the robot path data based on the tool path data generated in step (d); as well as (f) providing the generated robot path data via the communication interface.

2. The method of claim 1, wherein: The data indicating the geometric structure and color of each spatial substrate includes data representing each spatial substrate in three-dimensional space, in particular, a three-dimensional point cloud of each spatial substrate, and color data of each spatial substrate.

3. The method according to claim 1 or 2, wherein: The space substrate data provided includes - detecting, with the at least one computer processor, a user input indicative of a substrate classification associated with each spatial substrate and a user input indicative of a location of each spatial substrate within a workspace of the robot, - determining, using the at least one computer processor, substrate classification data for each spatial substrate and a position of each spatial substrate within a workspace of the robot based on detected user input, - providing data of the robot's workspace to the at least one computer processor via a communication interface, - determining, using the at least one computer processor, based on the provided data of the workspace of the robot, the collision geometry present in said workspace, - determining, using the at least one computer processor, scanning path data for a scanning path(s) to be followed by a scanning device along the surface of each spatial substrate based on the determined position of each spatial substrate within the workspace of the robot and the determined collision geometry, and providing the determined scanning path data to the scanning device via the communication interface, - Utilizing the at least one computer processor to generate spatial substrate data for each spatial substrate by acquiring data indicating the geometry and color of the spatial substrate acquired by the scanning device based on the provided scanning path data via the communication interface and combining the acquired data at least with the determined corresponding substrate classification data.

4. The method according to any one of claims 1 to 3, wherein: The data indicative of the identity of the at least one coating material includes a name of each coating material type, an ID of each coating material type, or a combination thereof.

5. The method according to any one of claims 1 to 4, wherein: The coating tool tolerance data includes target distance data, overlap percentage data, pattern size data, (multiple) rotational tolerances around the z-axis of the coating tool, (multiple) rotational tolerances around the x-axis of the coating tool, (multiple) rotational tolerances around the y-axis of the coating tool, or a combination thereof.

6. The method according to any one of claims 1 to 5, wherein: The rule set for coating outer edges and edges adjacent to open space(s) includes at least one algorithm for determining outer edges and at least one algorithm for determining edges adjacent to open space(s) present within the surface of the space substrate.

7. The method according to any one of claims 1 to 6, wherein: The set of rules for coating a major surface includes rules for determining the start of a coating procedure, rules for coating direction, rules for rotation of a coating tool within a tool path, rules for surface separation, or a combination thereof.

8. The method according to any one of claims 1 to 7, wherein: The at least one rule set for a spatial substrate type that matches the substrate classification data contained in the provided spatial substrate data includes at least one rule for coating edges adjacent to (multiple) open spaces of the corresponding type of spatial substrate, data regarding the required quality of the (multiple) tool paths, optionally at least one rule for coating (multiple) open spaces within the spatial substrate, and optionally at least one rotational tolerance for the coating tool.

9. The method according to any one of claims 1 to 8, wherein: The tool path data for generating the tool path(s) to be followed by the coating tool along the surface of each spatial substrate includes: - generating, with the at least one computer processor, a 3D model of each spatial substrate based on the provided spatial substrate data, and optionally applying a set of rules to smooth the surface of each generated 3D model, - using the computer processor to determine, based on the acquired coating process parameter data and the generated and optionally smoothed 3D model(s), the outer edge(s), the edge(s) adjacent to the open spaces, the major surface, and the open spaces(s) present within each space substrate, - generating, with the at least one computer processor, tool path data for outer edges and tool path data for edges adjacent to the open space(s) based on the obtained coating program data, the obtained coating tool parameter data, the obtained substrate type data, the determined outer edge(s), edge(s) adjacent to the open space(s) and the open space(s) and the generated and optionally smoothed 3D model(s), - generating, with the at least one computer processor, tool path data for the major surface of each spatial substrate based on the acquired coating procedure data, the acquired coating tool parameter data, the determined major surface and the generated and optionally smoothed 3D model(s), and - Optionally, repeating said steps for at least one further coating material based on the acquired coating program data and the acquired coating tool parameter data associated with the at least one further coating material.

10. The method according to any one of claims 1 to 9, wherein: Generating robot path data includes determining a collision geometry within a workspace of the robot based on the spatial substrate data, and determining the robot path data based on the determined collision geometry and the generated tool path data.

11. The method according to any one of claims 1 to 10, wherein: Generating robot path data further includes - ordering, using the at least one computer processor and prior to generating the robot path data, the generated tool path data such that the robot path(s) generated from the tool path data for the outer edges and edges adjacent to the open space(s) are executed before or after the robot path(s) generated from the tool path data for the primary surface, and / or - Optimizing the generated or sequenced robot path data using the computer processor.

12. A computing device for generating robot path data for a robot including a coating tool to follow during coating of a spatial substrate with at least one coating material, the computing device comprising: - at least one computer processor; as well as - a memory storing instructions which, when executed by the processor, configure the apparatus to perform the steps of any one of claims 1 to 11.

13. A robotic system for coating at least one surface of a space substrate with at least one coating material, the system comprising: - a computing device according to claim 12, for generating robot path data of a robot path(s) to be followed by a robot of the robot system during coating of the at least one surface of the spatial substrate with at least one coating material, - a robotic device configured to receive the generated robotic path data and to apply at least one coating material from a coating tool to at least a portion of the surface of the spatial substrate using the received robotic path data.

14. Use of the method according to any one of claims 1 to 11 or the computing device according to claim 12 for coating at least a portion of a surface of a space substrate with a coating material using a robotic system including a robot comprising a coating tool.

15. A non-transitory computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to perform the steps of the method according to any one of claims 1 to 11.

Images