Fluid coating equipment, fluid coating system, fluid coating method and program

Abstract

To provide fluid coating equipment, a fluid coating system, a fluid coating method and a program, which can reduce distortion of an image to be printed onto a printing surface even if the fluid coating equipment is inclined against the printing surface.SOLUTION: The fluid coating equipment ejects fluid from a discharge head onto a printing surface by dividing printing data into multiple printing images to scan the discharge head on the basis of each printing image against multiple printing regions. The fluid coating equipment comprises a first acquisition unit for acquiring distance information indicating a distance from a detector detecting the distance between the fluid coating equipment and the printing surface, a first calculation unit for calculating an inclination against the printing surface of a plane surface that is scanned by the discharge head from the distance information acquired with the first acquisition unit, and a correction unit for executing correction against the print image on the basis of the inclination calculated by the first calculation unit.SELECTED DRAWING: Figure 10

Description

Technical Field

[0001] The present invention relates to a liquid application device, a liquid application system, a liquid application method, and a program.

Background Art

[0002] In order to print an image larger than the printing area of a printing device, such as a road marking on a road, it is necessary to divide the print data into a plurality of print images and move the printing device so as to join the divided print images. As methods for estimating the self-position of this printing device, methods using GNSS (Global Navigation Satellite System) (for example, GPS (Global Positioning System), etc.), methods using odometry using the rotation speed of a motor, etc., methods using angular velocity and acceleration detection by an IMU (Inertial Measurement Unit), and methods for detecting landmarks using a camera are known. Further, as a method for finely adjusting the joint between the image printed at the first position and the images printed at the second and subsequent positions, a method of rotating the image using the angle detection result by self-position estimation is known.

[0003] As a technique for printing on such a road surface, a technique is disclosed in which the inclination of the road surface is calculated using a plurality of distance measurement sensors to determine the posture of the vehicle, and the distance from the spray nozzle for painting to the road surface is controlled to be constant, thereby making the line width and film thickness of the section lines drawn on the road surface uniform (for example, Patent Document 1).

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the conventional technology, even if the unevenness of the printing surface such as a road surface is detected and the discharge conditions are corrected, if the device tilts with respect to the printing surface, the landing position of the paint will shift, and there is a problem that distortion will occur in the printed image.

[0005] The present invention has been made in view of the above, and an object thereof is to provide a liquid application device, a liquid application system, a liquid application method, and a program that can reduce distortion of an image printed on a printing surface even when the liquid application device is inclined with respect to the printing surface.

Means for Solving the Problems

[0006] In order to solve the above-described problems and achieve the object, the present invention divides print data into a plurality of print images, scans a discharge head based on each of the print images for a plurality of print areas, and discharges liquid from the discharge head onto a printing surface. A liquid application device, comprising: a first acquisition unit that acquires distance information indicating the distance from a detection device that detects the distance from the printing surface; a first calculation unit that calculates an inclination of a plane scanned by the discharge head with respect to the printing surface from the distance information acquired by the first acquisition unit; and a correction unit that corrects the print image based on the inclination calculated by the first calculation unit.

Effects of the Invention

[0007] According to the present invention, even when the liquid application device is inclined with respect to the printing surface, distortion of an image printed on the printing surface can be reduced.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the liquid application device, the liquid application system, the liquid application method, and the program according to the present invention will be described in detail with reference to the drawings. Further, the present invention is not limited by the following embodiments, and the constituent elements in the following embodiments include those that can be easily conceived by those skilled in the art, substantially the same ones, and those within the so-called equivalent range. Furthermore, various omissions, substitutions, changes, and combinations of the constituent elements can be made without departing from the gist of the following embodiments.

[0010] (Overall Configuration of Liquid Application Device) FIG. 1 is a diagram showing an example of the overall configuration of a liquid coating apparatus according to an embodiment. With reference to FIG. 1, the overall configuration of the liquid coating apparatus 1 according to this embodiment will be described.

[0011] The liquid coating apparatus 1 shown in FIG. 1 is an apparatus that divides a wide liquid coating area on a road surface or a wall surface of a building or the like (hereinafter referred to as a "printing surface") into a plurality of printing areas and sequentially moves to each printing area, and divides printing data for printing on the liquid coating area into a plurality of printing images for printing. FIG. 1(a) is a side view of the liquid coating apparatus 1, and FIG. 1(b) is a plan view of the liquid coating apparatus 1 as viewed from above. Note that "printing" shall mean applying or spraying ink onto a road or a wall surface or the like.

[0012] As shown in FIG. 1, the liquid coating apparatus 1 includes a printing unit 21 and a control unit 22. Note that in the liquid coating apparatus 1, the printing unit 21 and the control unit 22 may each have a separate system configuration.

[0013] The printing unit 21 is a unit for discharging ink and performing printing on the printing surface while moving. Note that "ink" is a liquid that can be applied or sprayed onto the printing surface. As shown in FIG. 1, the printing unit 21 includes a carriage 2, a rail 3, a three-dimensional camera 7, a two-dimensional camera 8, a GNSS receiver 9, tires 10, a frame 11, a structure 12, an alignment member 13, and a lidar 14 (an example of a detection device).

[0014] The carriage 2 mounts an ink ejection head 2a to be described later, moves in the main scanning direction (arrow A shown in FIG. 1(b)) along the rail 3, and moves in the sub-scanning direction when the rail 3 moves in the sub-scanning direction (arrow B shown in FIG. 1(b)). Further, the carriage 2 also moves in the height direction (arrow C shown in FIG. 1(a)). The carriage 2 reciprocates in the main scanning direction along the rail 3 by a head movement mechanism 23 (see FIG. 9 to be described later) constituted by a belt, a pulley, a motor, and the like.

[0015] The rail 3 is a rail member that supports the carriage 2 for movement in the main scanning direction and is horizontally supported on the frame 11 so as to be movable in the sub-scanning direction. The rail 3 reciprocates on the frame 11 in the sub-scanning direction orthogonal to the main scanning direction by a rail movement mechanism 24 (see FIG. 9 described later) constituted by a belt, a pulley, a motor, and the like.

[0016] That is, the carriage 2 mounted with the ink ejection head 2a can freely move in the front-rear direction (sub-scanning direction) and the left-right direction (main scanning direction) on the plane surrounded by the frame 11 of the liquid application device 1. Note that the plane on which the carriage 2 moves (scans) in the sub-scanning direction and the main scanning direction may be hereinafter referred to as a "scanning plane".

[0017] The three-dimensional camera 7 is a three-dimensional shape measurement device for surrounding measurement that is supported by the front portion of the frame 11 and images the periphery of the liquid application device 1. The three-dimensional camera 7 may be powered from a battery mounted thereon, or may be powered from the power supply system 5 assuming continuous operation.

[0018] The two-dimensional camera 8 is an imaging device that images the printing surface and the vicinity of the printed image printed on the printing surface. Therefore, the imaging direction of the two-dimensional camera 8 is downward. The two-dimensional camera 8 transmits the captured imaging image to a controller unit 6 described later. The two-dimensional camera 8 may be powered from a battery mounted thereon, or may be powered from the power supply system 5 assuming continuous operation. Further, the printing unit 21 may be provided with an illumination device for imaging by the two-dimensional camera 8.

[0019] The GNSS receiver 9 is a receiving device that receives a positioning signal for measuring the current position on the earth from positioning satellites based on GNSS (for example, GPS, etc.). The GNSS receiver 9 transmits the received positioning signal to the controller unit 6.

[0020] The tire 10 is a member that is attached to the lower part of the frame 11 and rotates by being pushed by an operator or the like to move the printing unit 21. As a result, the liquid application device 1 can be moved in four directions: forward, backward, left, and right.

[0021] The frame 11 constitutes the base of the printing unit 21 and is a frame structure that supports the rail 3, the three-dimensional camera 7, the GNSS receiver 9, the structure 12, etc. from below.

[0022] The structure 12 is a structure assembled with pipes or the like that form the outer frame of the printing unit 21. The structure 12 is equipped with a two-dimensional camera 8 and a lidar 14 on its upper surface.

[0023] The alignment member 13 is a rod-shaped member supported on the side surface of the frame 11 and extending toward the front of the printing unit 21. By aligning the tip of the alignment member 13 with a mark or the like on the target printing surface, the movement of the liquid application device 1 can be guided.

[0024] The lidar 14 is a device that irradiates light in the search range and detects the distance to the target object (here, the printing surface) by detecting the reflected light from the target object. Here, the detection range of the distance by the lidar 14 is larger than the printing area where each printed image obtained by dividing the print data is printed. The lidar 14 transmits the detected distance information to the controller unit 6.

[0025] Note that the liquid application device 1 may be provided with distance measurement sensors or an IMU installed at the four corners of the printing unit 21 instead of or in addition to the lidar 14.

[0026] As shown in FIG. 1, the control unit 22 includes an ink supply system 4, a power supply system 5, and a controller unit 6.

[0027] The ink supply system 4 is a unit that supplies ink for printing to the ink ejection head 2a of the carriage 2 via a pipe 4a which is an ink flow path. In this embodiment, the ink supply system 4 is assumed to move following the printing unit 21, but it is not limited to this, and it may be self-propelled independently of the printing unit 21.

[0028] The power supply system 5 is a unit that supplies power for driving the control unit 22, the head movement mechanism 23, the ink ejection head 2a, etc.

[0029] The controller unit 6 is a control unit for controlling the operation of the liquid application device 1. The controller unit 6 controls, for example, the operations of the head movement mechanism 23, the rail movement mechanism 24, and the ink ejection head 2a, or estimates the position of the liquid application device 1 from the positioning signal received by the GNSS receiver 9. The controller unit 6 uses the captured image captured by the three-dimensional camera 7 mainly for avoiding contact with obstacles and the like using a method such as image correlation, or for estimating the movement amount and posture of the liquid application device 1. Further, the controller unit 6 calculates the inclination of the scanning surface of the printing unit 21 with respect to the printing surface based on the distance information from the printing surface detected by the lidar 14. Furthermore, the controller unit 6 stores the positioning signal received by the GNSS receiver 9 and the odometry information such as the cumulative movement amount of the liquid application device 1. Note that the liquid application device 1 may include a plurality of GNSS receivers 9, and the controller unit 6 may correct the position information from the positioning signals received by the plurality of GNSS receivers 9.

[0030] Note that the liquid application device 1 shown in FIG. 1 has been described as being moved mainly by manual pushing or the like by an operator, but it is not limited to this, and the liquid application device 1 may be a self-propelled device provided with a motor or the like capable of controlling the rotation of the tire 10.

[0031] (Overview of the overall operation of the liquid application device) FIG. 2 is a diagram for explaining an outline of operations when the liquid application apparatus according to the embodiment prints outside the scanning range of the carriage. FIG. 3 is a diagram for explaining an operation of moving the liquid application apparatus according to the embodiment using an alignment member when moving the liquid application apparatus. FIG. 4 is a diagram for explaining a state in which the liquid application apparatus is tilted with respect to the printing surface. FIG. 5 is a diagram for explaining a state in which the landing position of the ink is shifted due to the liquid application apparatus being tilted with respect to the printing surface. FIG. 6 is a diagram for explaining an outline of the rotation correction process of the liquid application apparatus according to the embodiment. FIG. 7 is a diagram for explaining the process of template matching. FIG. 8 is a diagram for explaining an outline of the distortion correction process of the liquid application apparatus according to the embodiment. With reference to FIGS. 2 to 8, an outline of the overall operation of the liquid application apparatus 1 according to the present embodiment will be described.

[0032] As described above, the liquid application apparatus 1 is movable in four directions: front, rear, left, and right. That is, the carriage 2 equipped with the ink ejection head 2a can freely move in four directions: front, rear, left, and right on the plane surrounded by the frame 11 of the liquid application apparatus 1. As a result, when the liquid application apparatus 1 causes the ink ejection head 2a to print on a printing surface outside the scanning range, as shown in FIG. 2, the entire liquid application area, which is the entire area to be printed on the printing surface, is divided into a plurality of printing areas, and the entire printing data to be printed on the liquid application area is divided into a plurality of printing images corresponding to each printing area. While moving the liquid application area in the front-rear direction and the left-right direction, each printing image is printed while connecting these joints so that the joints are less noticeable, thereby completing the printing of the entire image based on the printing data.

[0033] In addition, the liquid application device 1 according to the present embodiment can be manually moved back and forth, left and right by an operator by pushing it by hand or the like. When moving to each printing area, it moves using lines, marks, etc. drawn with chalk or the like as marks on the printing surface. Specifically, as shown in FIG. 3, the operator moves the tip of the alignment member 13 installed on the liquid application device 1 so as to match the mark made of chalk drawn on the printing surface, so that printing by ink ejection from the ink ejection head 2a is performed in each printing area. Thereby, the operator can perform rough alignment to each printing area when moving the liquid application device 1.

[0034] However, with regard to the alignment of the liquid application device 1 by the alignment member 13, it only provides rough alignment. In order to make the joints of the printed images printed in each printing area less conspicuous, after alignment by the alignment member 13, it is necessary to accurately calculate the amount of deviation (displacement amount, displacement angle) of the position of the liquid application device 1 and correct the printed image to be printed according to the deviation amount.

[0035] In addition, when the liquid application device 1 is moved back and forth and left and right by manual pushing by an operator or the like, the scanning surface may tilt with respect to the printing surface due to the unevenness of the printing surface. Fig. 4(a) shows the state of the liquid application device 1 when the printing surface RS is flat, and Fig. 4(b) shows a state where a stepped portion SP exists on the printing surface RS and the liquid application device 1 is tilted with respect to the printing surface RS because a part of the tire 10 of the liquid application device 1 has climbed onto the stepped portion SP. As shown in Fig. 4(b), when the scanning surface of the liquid application device 1 tilts with respect to the printing surface RS, a difference (distance difference) occurs between the distance to the printing surface detected by the rider 14 when there is no stepped portion SP and the distance to the printing surface detected by the rider 14 when the tire 10 climbs onto the stepped portion SP. Therefore, when the ink ejection head 2a of the carriage 2 ejects ink in the state where the scanning surface shown in Fig. 4(b) is tilted with respect to the printing surface RS such that such a distance difference occurs, as shown in Fig. 5, the landing position of the ink on the printing surface RS will deviate from the original landing position (that is, the case shown in Fig. 4(a)). As a result, the printed image printed by the liquid application device 1 will be distorted. Therefore, when the scanning surface of the liquid application device 1 is tilted with respect to the printing surface, it is necessary to correct the printed image before printing so that the distortion (skew) when printing the printed image is reduced.

[0036] Therefore, the liquid application device 1 according to the present embodiment calculates the amount of deviation (displacement amount, displacement angle) from the ideal position of the liquid application device 1 after moving the liquid application device 1 to the printing area to be printed next, and corrects (hereinafter, may be referred to as "rotation correction") the printed image corresponding to the printing area using the amount of deviation. Then, the liquid application device 1 calculates the inclination of the scanning surface with respect to the printing surface based on the distance information detected by the rider 14, and performs correction (hereinafter, may be referred to as "distortion correction") to reduce the distortion that may occur due to the inclination on the printed image corrected by rotation as described above, and then performs printing on the printing surface.

[0037] First, after moving the liquid application device 1 to the printing area to be printed next, in order to calculate the deviation amount (displacement amount, displacement angle) from the ideal position of the liquid application device 1, for example, based on the position estimated from the measurement signal received by the GNSS receiver 9, the position detected from the rotation direction and rotation amount of the tire 10, or the position of the liquid application device 1 detected and estimated by other sensors, etc., the deviation amount from the ideal position may be calculated. In the present embodiment, as an example of the operation of calculating the above deviation amount, by performing template matching on the captured image after movement using the template image on the captured image captured by the two-dimensional camera 8 before movement, the image portion that matches the template image is specified, and the details of the operation of specifying the deviation amount will be described later. Here, with reference to FIGS. 6 and 7, the outline of the calculation process of the displacement amount and displacement angle and the rotation correction process using template matching by the liquid application device 1 will be described. Note that, for simplicity of explanation in FIG. 6, the illustration of the liquid application device 1 is omitted. Also, in the explanation of FIG. 6, in order to explain the outline of the flow of the calculation process of the displacement amount and displacement angle using template matching, the explanation of distortion correction is omitted.

[0038] As an initial process, the liquid application device 1 divides a wide liquid application area on the printing surface into a plurality of printing areas, and divides the printing data for printing on the liquid application area into a plurality of printing images corresponding to each printing area. Note that it is not limited to the case where the printing data is divided into a plurality of printing images by the liquid application device 1. For example, the liquid application device 1 may receive and use a plurality of printing images in which the printing data is divided by an external device 30 via the I / F 63 described later. First, as shown in FIG. 6(a), when the liquid application device 1 moves to the initial position, it prints the printing image PIM1 on the printing area corresponding to the position. Here, as described above, when the liquid application device 1 moves, the operator moves the tip of the alignment member 13 so as to match the mark drawn on the printing surface. Then, the liquid application device 1 captures the captured image CIM1 with the two-dimensional camera 8 for the imaging area of the two-dimensional camera 8 including the printing image PIM1.

[0039] Next, as shown in FIG. 6(b), the liquid application device 1 extracts a template image TP1 for performing template matching on the captured image CIM1 in the captured image corresponding to the next printing area (captured image CIM2, which will be described later). Note that the extraction of the template image includes not only actually cutting out the template image as data from the captured image, but also specifying the area of the template image in the captured image. Here, the extraction of the template image TP1 in the captured image CIM1 is performed from the overlapping area between the captured image CIM1 and the captured image corresponding to the next printing area (captured image CIM2, which will be described later). This is to enable the template matching to identify the image portion in the captured image CIM2 that matches or is similar to the template image TP1 extracted from the captured image CIM1. Therefore, the imaging areas for imaging adjacent printing areas need to overlap (overlap) with each other. In the example shown in FIG. 6, since the liquid application device 1 is moved from left to right, the area near the right end of the captured image CIM1 overlaps with the area near the left side of the captured image CIM2 shown in FIG. 6(c), and the liquid application device 1 extracts the template image TP1 from the overlapping area in the captured image CIM1.

[0040] Next, the operator moves the liquid application device 1 to the printing area to be printed next. At this time, as described above, since the alignment by the alignment member 13 only provides rough alignment, it is difficult to make the connection between the printed area where the printed image PIM1 has already been printed and the moved printing area coincide without deviation. Therefore, the liquid application device 1 sequentially performs the following processes.

[0041] As shown in FIG. 6(c), after moving to the printing area to be printed next, the liquid application device 1 captures an imaging image CIM2 of the imaging area including the printing area with the two-dimensional camera 8. In this case, as described above, since the alignment by the alignment member 13 of the liquid application device 1 remains a rough alignment, the imaging image CIM2 shown in FIG. 6(c) is misaligned or rotated compared to the ideal position with respect to the imaging image CIM1 shown in FIG. 6(b). Therefore, the liquid application device 1 performs template matching to detect an image portion that coincides with or is similar to the template image TP extracted from the imaging image CIM1 on the imaging image CIM2.

[0042] Here, with reference to FIG. 7, the operation of template matching will be described. Template matching is a process of detecting an image portion in the image IM to be detected that matches or is similar to the target template image TP. In template matching, for example, it starts with comparing the upper left image portion of the image IM with the template image TP, calculates a similarity indicating the degree of similarity between the image portion and the template image TP, and determines whether the value indicating that the similarity is a value indicating match or similarity. This calculation and determination of the similarity are repeatedly performed while raster-scanning the template image TP over the image IM. As algorithms for template matching, for example, there are SSD (Sum of Squared Difference), SAD (Sum of Absolute Difference), NCC (Normalized Cross Correlation), ZNCC (Zero means Normalized Cross Correlation), etc., and at least any one of these may be used. Here, the similarity may be such that the higher the calculated value, the higher the degree of similarity, or the lower the value, the higher the degree of similarity, depending on the algorithm of template matching. In the present embodiment, it will be described that the higher the similarity value, the higher the degree of similarity. Also, in template matching, during raster scanning, when the calculated similarity exceeds a predetermined threshold, it may be determined that the image portion is an image portion that matches the template image TP, or the similarity corresponding to all image portions of the image IM is calculated, and if the maximum similarity among them exceeds the predetermined threshold, it may be determined that the image portion corresponding to the similarity is an image portion that matches the template image TP.

[0043] Returning to FIG. 6, the description will be continued. As described above, when the captured image CIM2 is displaced from the ideal position or rotated, even if template matching is performed while raster scanning the template image TP1 as it is on the captured image CIM2, it is impossible to detect an image portion that coincides with or is similar to the template image TP1 on the captured image CIM2. Therefore, the liquid application device 1 performs template matching using the template image TP on the rotated captured image CIM2 while rotating the captured image CIM2 by a predetermined angle. In order to rotate the captured image CIM2, for example, an affine transformation or the like may be used. As a result, as shown in FIG. 6(c), an image portion that coincides with or is similar to the template image TP1 can be detected on the captured image CIM2, and this image portion is set as the comparison image TP2. Then, when the position of the comparison image TP2 as an image portion that coincides with or is similar to the template image TP1 is determined on the captured image CIM2, since the positional relationship between the printing area in the captured image CIM1 and the template image TP1 is known, the ideal printing area to be printed in the captured image CIM2 (that is, the printing area in the captured image CIM2 where the joint with the printing area of the captured image CIM1 coincides) corresponding to the position of the comparison image TP2 in the captured image CIM2 is determined. Then, the liquid application device 1 calculates the deviation amount (displacement amount, displacement angle) of the above-described ideal printing area from the printing area that is the original printing target determined within the captured image CIM2. Thereby, it is possible to highly accurately detect the relative position and the deviation amount of rotation before and after the movement of the liquid application device 1 for printing.

[0044] In the above-described template matching, it has been assumed that the template matching is performed while rotating the captured image CIM2, but the present invention is not limited to this, and the template matching may be performed while rotating the template image TP1 with the captured image CIM2 as it is. That is, the template matching may be performed while changing the relative angle between the captured image CIM2 and the template image TP1.

[0045] Next, based on the calculated deviation amount (displacement amount · displacement angle), the liquid application device 1 performs rotational correction to move and rotate the print image PIM2 shown in Fig. 6(d) to be printed next as shown in Fig. 6(e), and generates a print image PIM2a. Then, as shown in Fig. 6(f), the liquid application device 1 performs printing on the printing surface using the corrected print image PIM2a. As a result, as shown in Fig. 6(g), the joint between the print image PIM1 printed on the previous printing surface and the print image PIM2a printed this time can be made to substantially coincide. In this way, by performing printing with a print image corrected based on the calculated deviation amount (displacement amount · displacement angle), the joints of the printed print images can be made inconspicuous. However, as described above, when the scanning surface is inclined with respect to the printing surface because the liquid application device 1 rides over a stepped portion or the like of the printing surface, a distorted image will be printed on the printing surface if the rotation-corrected print image is printed as it is. Here, the outline of the distortion correction by the liquid application device 1 will be described with reference to Fig. 8.

[0046] As described above, when the scanning surface is inclined with respect to the printing surface because the liquid application device 1 rides over a stepped portion or the like of the printing surface, the landing position of the ink on the printing surface will deviate from the original landing position. Therefore, if the rotation-corrected print image described above is printed as it is, a distorted pre-correction print image PIM_B as shown in Fig. 8 will be printed. Therefore, the liquid application device 1 first calculates the inclination of the scanning surface with respect to the printing surface based on the distance information detected by the lidar 14, and calculates a correction amount (skew correction amount shown in Fig. 8) for reducing the distortion. Then, as shown in Fig. 8, the liquid application device 1 performs distortion correction on the pre-correction print image PIM_B using the calculated correction amount, so that a post-correction print image PIM_A with reduced distortion is printed. Thereby, even if the liquid application device 1 is inclined with respect to the printing surface, the distortion of the image printed on the printing surface can be reduced.

[0047] (Hardware Configuration of Liquid Application Device) FIG. 9 is a diagram showing an example of the hardware configuration of the liquid application device according to the embodiment. With reference to FIG. 9, the hardware configuration of the liquid application device 1 according to the present embodiment will be described.

[0048] As shown in FIG. 9, the liquid application device 1 includes a carriage 2, a controller unit 6, a three-dimensional camera 7, a two-dimensional camera 8, a GNSS receiver 9, a lidar 14, an operation panel 15, a head movement mechanism 23, and a rail movement mechanism 24.

[0049] The carriage 2 is equipped with an ink ejection head 2a that ejects ink onto the printing surface, moves in the main scanning direction along the rail 3 shown in FIG. 1, and moves in the sub-scanning direction as the rail 3 moves in the sub-scanning direction. Further, the carriage 2 also moves in the height direction.

[0050] The controller unit 6 includes a CPU (Central Processing Unit) 61, a memory 62, an I / F 63, and a unit control circuit 64.

[0051] The CPU 61 is an arithmetic device that comprehensively controls the operation of the liquid application device 1. The CPU 61 performs data communication with the memory 62, the I / F 63, and the unit control circuit 64 via a bus. Further, the CPU 61 performs drive control of the head movement mechanism 23, the rail movement mechanism 24, and the ink ejection head 2a via the unit control circuit 64, and estimates the self-position of the liquid application device 1 from the positioning signal received by the GNSS receiver 9.

[0052] The memory 62 is a storage medium such as a ROM (Read Only Memory) or a RAM (Random Access Memory) that stores programs used for driving the CPU 61. Further, the memory 62 is used as a work area for the CPU 61.

[0053] I / F63 is a communication interface for connecting various external devices 30 such as tablet terminals, smartphones, PCs (Personal Computers), servers, and notebook PCs.

[0054] The unit control circuit 64 is a control circuit that controls the operations of the head movement mechanism 23, the rail movement mechanism 24, and the ink ejection head 2a according to the control by the CPU 61.

[0055] The three-dimensional camera 7 and the two-dimensional camera 8 each transmit the captured imaging images to the CPU 61 of the controller unit 6.

[0056] The GNSS receiver 9 receives a positioning signal for measuring the current position on the earth from positioning satellites based on GNSS (such as GPS, etc.), and transmits the positioning signal to the CPU 61 of the controller unit 6.

[0057] The lidar 14 irradiates light in the search range, detects the reflected light from the object (here, the printing surface), thereby detects the distance to the object, and transmits the detected distance information to the controller unit 6.

[0058] The operation panel 15 is a device that accepts operations on the liquid application device 1 and displays processing results, etc. by the liquid application device 1.

[0059] The head movement mechanism 23 is composed of a belt, a pulley, a motor, etc., and is a mechanism that reciprocates the carriage 2 along the rail 3 in the main scanning direction according to the control by the unit control circuit 64.

[0060] The rail movement mechanism 24 is composed of a belt, a pulley, a motor, etc., and is a mechanism that reciprocates the rail 3 on the frame 11 in the sub-scanning direction orthogonal to the main scanning direction according to the control by the unit control circuit 64. As a result, the carriage 2 supported by the rail 3 reciprocates in the sub-scanning direction.

[0061] Note that the hardware configuration of the liquid application apparatus 1 shown in FIG. 9 is an example, and it may be provided with other constituent devices.

[0062] (Configuration and Operation of Functional Blocks of Liquid Application Apparatus) FIG. 10 is a diagram showing an example of the configuration of functional blocks of the liquid application apparatus according to the embodiment. FIG. 11 is a diagram for explaining the relationship before and after correction in the rotation correction process of the liquid application apparatus according to the embodiment. FIG. 12 is a diagram for explaining the operation of determining the printing start position after correction in the rotation correction process of the liquid application apparatus according to the embodiment. With reference to FIGS. 10 to 12, the configuration and operation of the functional blocks of the liquid application apparatus 1 according to the present embodiment will be described.

[0063] As shown in FIG. 10, the liquid application apparatus 1 includes an imaging unit 101, an acquisition unit 102 (second acquisition unit), a direction specifying unit 103, a storage unit 104, a distance acquisition unit 105 (first acquisition unit), an image processing unit 110, and an image generation unit 120.

[0064] The imaging unit 101 is a functional unit that images each imaging region including each printing region where the liquid application apparatus 1 has moved. As a result, the imaging unit 101 obtains an imaging image obtained by imaging the imaging region. The imaging unit 101 is realized by the two-dimensional camera 8 shown in FIG. 9.

[0065] The acquisition unit 102 is a functional unit that acquires the imaging image imaged by the imaging unit 101. The acquisition unit 102 is realized, for example, by the CPU 61 shown in FIG. 9 executing a program.

[0066] The direction specifying unit 103 is a functional unit that specifies the moving direction of the liquid application device 1. The direction specifying unit 103 may, for example, specify a fixed direction (for example, the forward direction which is the imaging direction of the three-dimensional camera 7 shown in FIG. 1), may specify the moving direction according to an operation on the liquid application device 1 by an operator, may specify it by detecting the direction and rotation direction of the tire 10, or may specify the moving direction of the position of the liquid application device 1 estimated from the positioning signal received by the GNSS receiver 9. The direction specifying unit 103 is realized, for example, by a program being executed by the CPU 61 shown in FIG. 9.

[0067] The storage unit 104 is a functional unit that stores information such as print data, a plurality of printed images obtained by dividing the print data, the amount of displacement and the displacement angle calculated by the image processing unit 110, and the inclination of the scanning surface with respect to the printing surface. The storage unit 104 is realized by the memory 62 shown in FIG. 9.

[0068] The distance acquisition unit 105 is a functional unit that acquires distance information to the printing surface detected by the lidar 14. The distance acquisition unit 105 is realized, for example, by a program being executed by the CPU 61 shown in FIG. 9.

[0069] Note that the distance acquisition unit 105 may acquire information detected by a distance measuring sensor (an example of a detection device) or an IMU (an example of a detection device) provided instead of or in addition to the lidar 14 as described above.

[0070] The image processing unit 110 is a functional unit that calculates, for example, the amount of deviation (displacement amount and displacement angle) from an ideal position by performing image processing on the captured image captured by the imaging unit 101, and rotation-corrects the printed image corresponding to the printing area based on the amount of deviation. Further, the image processing unit 110 calculates the inclination of the scanning surface with respect to the printing surface based on the distance information acquired by the distance acquisition unit 105, and performs distortion correction on the rotation-corrected printed image based on the inclination.

[0071] Regarding the displacement amount (displacement quantity and displacement angle) based on the captured image, it will be described with reference to FIG. 11. When the liquid application device 1 can be moved to an ideal position as the next movement position by using the alignment member 13, the ideal imaging region captured by the imaging unit 101 is defined as the imaging region CAR_I, and the printing region determined by the imaging region CAR_I is defined as the printing region PAR_I. That is, the printing region that is continuously connected without displacement to the printing region where the printed image was printed before movement is the printing region PAR_I. Therefore, if the corresponding printed image is printed on the printing region PAR_I by the liquid application device 1, printing can be performed in a manner where the joint with the immediately preceding printed image is not conspicuous. However, in reality, since it is difficult to move the liquid application device 1 to an ideal position, as shown in FIG. 11, the actually captured imaging region CAR_R by the imaging unit 101 has a displacement in position and inclination compared to the ideal imaging region CAR_I. The printing region corresponding to the imaging region CAR_R in that case is the printing region PAR_R. Therefore, the image processing unit 110 calculates, by image processing including template matching for the captured image captured by the imaging unit 101, as shown in FIG. 11, the displacement angle Dθ which is the inclination displacement amount of the printing region PAR_R with respect to the printing region PAR_I, and calculates the displacement amounts Dx and Dy (hereinafter, may be referred to as the displacement amount (Dx, Dy)) which are the position displacement amounts of the printing region PAR_R with respect to the printing region PAR_I. Here, in FIG. 11, the printing region obtained by rotating the printing region PAR_R by the displacement angle Dθ is shown as the printing region PAR_P. And the printing region obtained by moving the printing region PAR_P by the displacement amount (Dx, Dy) coincides with the printing region PAR_I.

[0072] As shown in FIG. 10, the image processing unit 110 includes a displacement amount calculation unit 111 (second calculation unit), an inclination calculation unit 112 (first calculation unit), a printed image correction unit 113 (correction unit), and a printing start position determination unit 114 (determination unit). The image processing unit 110 is realized, for example, by a program being executed by the CPU 61 shown in FIG. 9.

[0073] The displacement amount calculation unit 111 is a functional unit that extracts a template image from a captured image obtained by capturing a printed printed image, and calculates the deviation amount (displacement amount, displacement angle) of an ideal printing area from the printing area that is originally the printing target by performing template matching on the captured image after the movement of the liquid application device 1.

[0074] First, the displacement amount calculation unit 111 extracts a template image for use in template matching from the captured image captured by the image processing unit 110 corresponding to the printing area where the printed image is printed by the imaging unit 120. For example, as shown in FIG. 12, the displacement amount calculation unit 111 extracts an image portion adjacent to the image portion corresponding to the printing area PAR1 in the captured image corresponding to the specific imaging area CAR1 (that is, the image portion including the printed image PIM11) as the template image TP11. In this case, the displacement amount calculation unit 111 is an image portion adjacent to the end portion (side) on the movement direction side of the liquid application device 1 specified by the direction specifying unit 103 in the image portion corresponding to the printing area PAR1 in the captured image corresponding to the specific imaging area CAR1 (that is, the image portion including the printed image PIM11), and, as described above, an image portion included in the overlap area between the captured images is extracted as the template image TP11. In this way, when the template image is extracted based on the movement direction specified by the direction specifying unit 103, the range for extracting the template image can be limited, so that the calculation time can be shortened.

[0075] In addition, when extracting a template image from an overlap region in a specific captured image, it is not necessarily limited to extracting, as the template image, an image portion close to the image portion corresponding to the printed area in the captured image. However, it is desirable that the template image extracted from the overlap region does not include the printed image printed on the printed area (printing region). This is because the color or drying state of the printed image printed on the printing surface (printing region) changes over time, and when template matching is performed, there is a possibility that an image portion that matches the template image including a part of the printed image cannot be detected from the captured image that is the target of the template matching.

[0076] Further, the template image extracted by the displacement amount calculation unit 111 does not necessarily have to be a single image, and a plurality of template images may be extracted and template matching may be performed using the plurality of template images.

[0077] Next, the displacement amount calculation unit 111 performs template matching on the captured image (hereinafter sometimes referred to as the post-movement captured image) (second captured image) captured by the imaging unit 101 for the printing area (target printing area) to be printed next by the liquid application device 1, using the template image extracted from the captured image before the movement of the liquid application device 1 (hereinafter sometimes referred to as the pre-movement captured image) (first captured image). In this case, the displacement amount calculation unit 111 performs template matching using the template image on the rotated post-movement captured image while rotating the post-movement captured image by a predetermined rotation amount each time by affine transformation or the like. Here, in the template matching, the displacement amount calculation unit 111 calculates the similarity between the image portion of the post-movement captured image, which compares whether it matches or is similar while raster-scanning the template image on the post-movement captured image, and the template image. Then, the displacement amount calculation unit 111 determines, for example, that the image portion where the calculated similarity is equal to or greater than a predetermined threshold is an image portion that matches or is similar to the template image. For example, in FIG. 12, an example is shown in which a comparison image TP12 is detected as an image portion that matches or is similar to the template image TP11 extracted from the pre-movement captured image (captured image in the imaging area CAR1) in the post-movement captured image (captured image in the imaging area CAR2) rotated by the displacement amount calculation unit 111.

[0078] Then, when the displacement amount calculation unit 111 detects an image portion in the post-movement captured image that matches or is similar to the template image, it calculates the rotation amount by which the post-movement captured image is rotated as the displacement angle Dθ. Also, as described above, when the position of the image portion in the post-movement captured image that matches or is similar to the template image is determined, since the positional relationship between the printing area in the pre-movement captured image and the template image is known, the ideal printing area to be printed in the post-movement captured image with respect to the position of the said image portion in the post-movement captured image (that is, the printing area in the post-movement captured image where the joint with the printing area of the pre-movement captured image matches) is determined. In the example of FIG. 12, when the position of the comparison image TP12, which is an image portion in the captured image (post-movement captured image) of the imaging area CAR2 that matches or is similar to the template image TP11, is determined, since the positional relationship between the printing area PAR1 in the pre-movement captured image and the template image TP11 is known, the ideal printing area PAR2 to be printed in the captured image (post-movement captured image) of the imaging area CAR2 with respect to the position of the comparison image TP12 in the captured image of the imaging area CAR2 is determined. Then, the displacement amount calculation unit 111 calculates the displacement amount (Dx, Dy) of the above-described ideal printing area PAR2 from the original printing area determined within the captured image (post-movement captured image) of the imaging area CAR2. In this case, the displacement amount calculation unit 111 can calculate the displacement amount of the center of the printing area PAR2 from the center of the original printing area (the center of the imaging area CAR2) determined within the captured image (post-movement captured image) of the imaging area CAR2 as the displacement amount (Dx, Dy). The displacement angle Dθ and the displacement amount (Dx, Dy) calculated by the displacement amount calculation unit 111 are calculated as the relative deviation before and after the movement of the liquid application device 1.

[0079] The inclination calculation unit 112 is a functional unit that calculates the inclination of the scanning surface with respect to the printing surface based on the distance information to the printing surface acquired by the distance acquisition unit 105. As described above, since the detection range of the distance to the printing surface by the lidar 14 is larger than the printing area where each printed image obtained by dividing the print data is printed, the inclination calculation unit 112 can accurately calculate the inclination of the scanning surface with respect to the printing surface, and the accuracy of distortion correction by the printing image correction unit 113 described later is also improved.

[0080] The printed image correction unit 113 is a functional unit that corrects the printed image to be printed at the position where the post-movement captured image of the liquid application device 1 is captured based on the displacement amount (displacement amount and displacement angle) calculated by the displacement amount calculation unit 111 and the inclination of the scanning surface with respect to the printed surface calculated by the inclination calculation unit 112. Specifically, the printed image correction unit 113 uses the displacement amount (displacement amount and displacement angle) calculated by the displacement amount calculation unit 111 to perform rotational correction (second correction) on the printed image to be printed at the position where the post-movement captured image of the liquid application device 1 is captured. Then, the printed image correction unit 113 calculates the amount of distortion (skew amount) that may occur due to the inclination based on the inclination of the scanning surface with respect to the printed surface calculated by the inclination calculation unit 112 for the rotationally corrected printed image. Then, the printed image correction unit 113 performs distortion correction (first correction) on the printed image so as to reduce the calculated skew amount to less than a predetermined value, and reduces the possible distortion. That is, the printed image correction unit 113 performs distortion correction after performing rotational correction.

[0081] Note that, in order to perform rotational correction on the printed image, a region (rotationally correctable region) that is one size larger than the printed region is required. Therefore, the upper limit value of the rotational correction is defined within the range where the printed region fits within this rotationally correctable region, and by moving the liquid application device 1 so that it falls within the upper limit value, it can be confirmed that the position of the liquid application device 1 is correct.

[0082] The printing start position determination unit 114 is a functional unit that determines the printing start position after the movement of the liquid application device 1 based on the ideal printing region to be printed in the post-movement captured image determined by the template matching by the displacement amount calculation unit 111. In the example of FIG. 12, the printing start position determination unit 114 determines the start position S for printing the printed image that has been rotationally corrected and distortion-corrected by the printed image correction unit 113 based on the ideal printing region PAR2 to be printed in the captured image (post-movement captured image) of the imaging region CAR2 determined by the template match by the inclination calculation unit 112.

[0083] Note that the printing start position determination unit 114 is not limited to determining the printing start position based on the printing area to be printed in the post-movement captured image determined by the template matching by the displacement amount calculation unit 111 as described above. For example, the printing start position determination unit 114 may display the image captured by the two-dimensional camera 8 on the operation panel 15, and determine the position designated by the operator via the operation panel 15 as the printing start position.

[0084] Then, the imaging unit 120 performs printing on the printing surface using the printed image corrected by the printed image correction unit 113. As a result, the joint between the printed image printed on the previous printing surface and the printed image printed this time can be made to substantially coincide. In this way, by performing printing using the printed image corrected (rotation correction, distortion correction) based on the displacement amounts (Dx, Dy) and displacement angle Dθ calculated by the displacement amount calculation unit 111 and the inclination calculated by the inclination calculation unit 112, the joints of the printed images printed can be made inconspicuous.

[0085] Note that at least any one of the functional units of the above-described image processing unit 110 may be realized by an external device 30 instead of the liquid application device 1. As a result, the processing load on the liquid application device 1 can be reduced. In this case, the liquid application device 1 and the external device 30 are configured as an example of a liquid application system.

[0086] The image forming unit 120 is a functional unit that performs printing of the print image on the printing surface by controlling the ejection of ink from the ink ejection head 2a starting from the print start position determined by the print start position determination unit 114 based on the print image corrected by the print image correction unit 113 while moving the carriage 2 in the main scanning direction and the sub-scanning direction. The image forming unit 120 is realized by the unit control circuit 64 shown in FIG. 9. Note that although the unit control circuit 64 is shown as a hardware circuit in FIG. 9, it is not limited thereto and may be realized by executing a program by the CPU 61. In this case, the image forming unit 120 is realized by executing a program by the CPU 61 shown in FIG. 9. As shown in FIG. 10, the image forming unit 120 includes a movement control unit 121 and an ejection control unit 122.

[0087] The movement control unit 121 is a functional unit that controls the reciprocating movement of the carriage 2 on which the ink ejection head 2a is mounted in the main scanning direction and the sub-scanning direction by controlling the head movement mechanism 23 and the rail movement mechanism 24. The movement control unit 121 moves the carriage 2 based on the print image corrected by the print image correction unit 113 and the start position determined by the print start position determination unit 114.

[0088] The ejection control unit 122 is a functional unit that controls the ink ejection operation of the ink ejection head 2a. The ejection control unit 122 ejects ink from the ink ejection head 2a based on the print image corrected by the print image correction unit 113.

[0089] Note that some or all of the functional units of the acquisition unit 102, the direction identification unit 103, the distance acquisition unit 105, the displacement amount calculation unit 111, the inclination calculation unit 112, the print image correction unit 113, and the print start position determination unit 114 may be realized by a hardware circuit (integrated circuit) such as an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit) instead of a program which is software.

[0090] In addition, each functional unit of the liquid application device 1 shown in FIG. 10 conceptually shows the functions and is not limited to such a configuration. For example, a plurality of functional units illustrated as independent functional units in the liquid application device 1 shown in FIG. 10 may be configured as one functional unit. On the other hand, the function of one functional unit in the liquid application device 1 shown in FIG. 10 may be divided into a plurality of functions and configured as a plurality of functional units. Further, each functional unit of the liquid application device 1 does not necessarily have to be configured as a clear software module as the block shown in FIG. 10, and it is sufficient that the function of each functional unit is realized as a whole when a program is executed in the liquid application device 1.

[0091] (Overall operation of the liquid application device) FIG. 13 is a flowchart showing an example of the flow of the overall operation of the liquid application device according to the embodiment. The flow of the overall operation of the liquid application device 1 according to the present embodiment will be described with reference to FIG. 13.

[0092] <Step S11> As an initial process, the image processing unit 110 of the liquid application device 1 divides a wide liquid application area of a printing surface such as a road into a plurality of printing areas, and divides the printing data for printing on the liquid application area into a plurality of printing images corresponding to each printing area. Then, the process proceeds to step S12.

[0093] <Step S12> The operator moves the liquid application device 1 to the initial position so that the tip of the alignment member 13 of the liquid application device 1 matches the mark drawn on the printing surface. In this case, the direction specifying unit 103 of the liquid application device specifies the moving direction of the liquid application device 1. Then, the process proceeds to step S13.

[0094] <Step S13> The imaging unit 101 of the liquid application device 1 images the imaging range including the printing area of the printing surface to obtain an imaging image. Then, the acquisition unit 102 of the liquid application device 1 acquires the pre-movement imaging image captured by the imaging unit 101. In this way, by using the pre-captured imaging image, for example, it can be confirmed visually that there are no foreign matters (such as dust, leaves, etc.) in the imaging image. Note that the imaging image may be used for adjusting the light amount of the lighting device used when the imaging unit 101 captures the image. Then, the process proceeds to step S14.

[0095] <Step S14> The distance acquisition unit 105 of the liquid application device 1 acquires the distance information to the printing surface detected by the lidar 14. The distance acquisition unit 105 sends the acquired distance information to the image processing unit 110. Then, the process proceeds to step S15.

[0096] <Step S15> The inclination calculation unit 112 of the liquid application device 1 calculates the inclination of the scanning surface with respect to the printing surface based on the distance information to the printing surface acquired by the distance acquisition unit 105. Then, the process proceeds to step S16.

[0097] <Step S16> The printing image correction unit 113 of the liquid application device 1 performs distortion correction to reduce the distortion that may occur due to the inclination on the printing image printed at the initial position based on the inclination of the scanning surface with respect to the printing surface calculated by the inclination calculation unit 112. Then, the process proceeds to step S17.

[0098] <Step S17> The image forming unit 120 of the liquid application device 1 moves the carriage 2 in the main scanning direction and the sub-scanning direction, and based on the printing image corrected by the printing image correction unit 113, controls the ejection of ink from the ink ejection head 2a to print the printing image on the printing surface. Then, the process proceeds to step S18.

[0099] <Step S18> The imaging unit 101 of the liquid application device 1 images an imaging range including a printed area where a printed image is printed by the image forming unit 120, and obtains an imaged image (pre-movement imaged image). Then, the acquisition unit 102 of the liquid application device 1 acquires the pre-movement imaged image imaged by the imaging unit 101. And the displacement amount calculation unit 111 of the liquid application device 1 extracts a template image used for template matching from the pre-movement imaged image acquired by the acquisition unit 102. In this case, the displacement amount calculation unit 111 extracts the template image from the pre-movement imaged image based on the moving direction of the liquid application device 1 specified by the direction specifying unit 103. In this way, by using the imaged image (pre-movement imaged image) obtained by imaging the imaging range including the printed area after printing instead of before the printed image is printed by the image forming unit 120, it is possible to suppress the error caused by the displacement of the position of the liquid application device 1 due to vibration or the like during the printing operation by the image forming unit 120.

[0100] Note that the extraction target of the template image is the pre-movement imaged image imaged after printing as described above, but it is not limited to this. That is, as long as the positional deviation due to the printing operation as described above is allowable, the template image may be extracted from the imaged image imaged before printing in the printed area.

[0101] Then, it proceeds to step S19.

[0102] <Step S19> The operator moves the liquid application device 1 to the printing area where the liquid application device 1 will print next so that the tip of the alignment member 13 of the liquid application device 1 aligns with the mark drawn on the printing surface. Then, it proceeds to step S20.

[0103] <Step S20> Then, the imaging unit 101 images the imaging range including the moved printing area to obtain an imaging image (post-movement imaging image). The acquisition unit 102 acquires the post-movement imaging image captured by the imaging unit 101. Note that, based on the captured imaging image, for example, it can be confirmed visually that there are no foreign objects (such as dust, leaves, etc.) in the imaging image. Also, the imaging image may be used for adjusting the light amount of the lighting device used during imaging by the imaging unit 101. Then, the process proceeds to step S21.

[0104] <Step S21> The distance acquisition unit 105 acquires distance information to the printing surface detected by the lidar 14. The distance acquisition unit 105 sends the acquired distance information to the image processing unit 110. Then, the process proceeds to step S22.

[0105] <Step S22> The displacement amount calculation unit 111 performs template matching on the post-movement imaging image acquired by the imaging unit 101 using the template image extracted from the pre-movement imaging image. Then, when the displacement amount calculation unit 111 detects an image portion in the post-movement imaging image that matches or is similar to the template image according to the above-described procedure, the rotation amount by which the post-movement imaging image is rotated is calculated as the displacement angle Dθ. And the displacement amount calculation unit 111 calculates the displacement amount (Dx, Dy) of the ideal printing area (that is, the printing area in the post-movement imaging image where the connection with the printing area of the pre-movement imaging image matches) from the original printing area determined within the post-movement imaging image according to the above-described procedure. The displacement angle Dθ and the displacement amount (Dx, Dy) calculated by the displacement amount calculation unit 111 are calculated as the relative displacement before and after the movement of the liquid application device 1. Then, the process proceeds to step S23.

[0106] <Step S23> The printed image correction unit 113 performs rotational correction on the printed image to be printed at the position where the post-movement imaging image of the liquid application device 1 is captured using the displacement amount (displacement amount (Dx, Dy) · displacement angle Dθ) calculated by the displacement amount calculation unit 111. Then, the process proceeds to step S24.

[0107] <Step S24> Based on the distance information to the printing surface acquired by the distance acquisition unit 105 in step S21, the inclination calculation unit 112 calculates the inclination of the scanning surface with respect to the printing surface. Then, it proceeds to step S25.

[0108] <Step S25> Based on the inclination of the scanning surface with respect to the printing surface calculated by the inclination calculation unit 112, the printing image correction unit 113 performs distortion correction to reduce the distortion that may occur due to the inclination for the printing image rotationally corrected in step S23. Then, it proceeds to step S26.

[0109] <Step S26> Then, based on the ideal printing area to be printed in the post - movement captured image determined by the template matching by the inclination calculation unit 112, the printing start position determination unit 114 of the liquid application device 1 determines the post - movement printing start position of the liquid application device 1. Then, it proceeds to step S27.

[0110] <Step S27> While moving the carriage 2 in the main scanning direction and the sub - scanning direction, based on the printing image corrected by the printing image correction unit 113, the imaging unit 120 performs ejection control of the ink of the ink ejection head 2a from the printing start position determined by the printing start position determination unit 114, thereby printing the printing image on the printing surface. Then, it proceeds to step S28.

[0111] <Step S28> The image processing unit 110 checks whether printing has been completed up to the last printing area for all the divided printing areas. If printing has not been completed up to the last printing area (step S28: No), it returns to step S18. If printing has been completed up to the last printing area (step S28: Yes), the overall operation (printing operation) of the liquid application device 1 is terminated.

[0112] As described above, in the liquid application apparatus 1 according to the present embodiment, the print data is divided into a plurality of print images, and the ink ejection head 2a is scanned based on each of the print images for a plurality of print areas to eject liquid from the ink ejection head 2a onto the print surface. The distance acquisition unit 105 acquires distance information indicating the distance from the lidar 14 that detects the distance to the print surface. The inclination calculation unit 112 calculates the inclination of the plane (scanning plane) scanned by the ink ejection head 2a with respect to the print surface from the distance information acquired by the distance acquisition unit 105. The print image correction unit 113 corrects the print image based on the inclination calculated by the inclination calculation unit 112. Specifically, the print image correction unit 113 performs distortion correction to reduce the distortion caused by the inclination based on the inclination calculated by the inclination calculation unit 112. As a result, even if the liquid application apparatus is inclined with respect to the print surface, the distortion of the image printed on the print surface can be reduced.

[0113] Further, in the liquid application apparatus 1 according to the present embodiment, the detection range of the distance to the print surface by the lidar 14 is set to be larger than the print area. As a result, the inclination calculation unit 112 can accurately calculate the inclination of the scanning plane with respect to the print surface, and the accuracy of the distortion correction by the print image correction unit 113 is also improved.

[0114] Also, in the liquid application device 1 according to the present embodiment, the acquisition unit 102 acquires a pre-movement captured image captured by the imaging unit 101 before the movement of the liquid application device 1 to the target printing area, and a post-movement captured image captured by the imaging unit 101 after the movement of the liquid application device 1 to the target printing area. The displacement amount calculation unit 111 extracts a template image from the pre-movement captured image, performs template matching using the template image on the post-movement captured image, and based on the position of the image portion in the post-movement captured image that matches or is similar to the template image detected by the template matching, calculates the relative displacement amount (displacement amount and displacement angle) before and after the movement of the liquid application device 1. The printed image correction unit 113 performs rotation correction for moving and rotating the printed image to be printed on the target printing area based on the displacement amount calculated by the displacement amount calculation unit 111. As a result, it becomes possible to print while joining each printed image so that these joints are less conspicuous.

[0115] Also, in the liquid application device 1 according to the present embodiment, the printing start position determination unit 114 determines the start position of the printing area based on the printing area in the post-movement captured image determined by the template matching by the displacement amount calculation unit 111. As a result, it becomes possible to start printing each printed image so that these joints are less conspicuous.

[0116] In the above-described embodiment, when at least any one of the functions of the liquid application device 1 is realized by executing a program, the program is provided by being pre-installed in a ROM or the like. Further, in the above-described embodiment, the program executed by the liquid application device 1 may be configured to be recorded and provided on a computer-readable recording medium such as a CD-ROM (Compact Disc Read Only Memory), a flexible disk (FD), a CD-R (Compact Disk-Recordable), or a DVD (Digital Versatile Disc) in an installable format or an executable format file. Further, in the above-described embodiment, the program executed by the liquid application device 1 may be configured to be stored on a computer connected to a network such as the Internet and downloaded via the network for providing. Further, in the above-described embodiment, the program executed by the liquid application device 1 may be configured to be provided or distributed via a network such as the Internet. Further, in the above-described embodiment, the program executed by the liquid application device 1 has a module configuration including at least any one of the above-described functional units, and as actual hardware, the CPU reads the program from the above-described storage device and executes it, so that the above-described functional units are loaded and generated on the main storage device.

[0117] Aspects of the present invention are as follows. <1>A liquid application device that divides print data into a plurality of print images and scans a discharge head to discharge liquid onto a print surface for each of the plurality of print areas based on each of the print images, a first acquisition unit that acquires distance information indicating the distance from a detection device that detects the distance from the print surface; a first calculation unit that calculates the inclination of the plane scanned by the discharge head with respect to the print surface from the distance information acquired by the first acquisition unit; a correction unit that corrects the print image based on the inclination calculated by the first calculation unit; A liquid application device comprising the same. <2>The liquid application device according to <1> above, wherein the correction unit performs first correction to reduce distortion caused by the inclination based on the inclination calculated by the first calculation unit. <3>The liquid application device according to <2> above, wherein the correction unit calculates a skew amount when the printed image is printed from the inclination calculated by the first calculation unit, and performs the first correction so as to reduce the skew amount of the printed image. <4>The liquid application device according to any one of <1> to <3> above, wherein a detection range of the distance from the detection device to the printing surface is larger than the printing area. <5>A second acquisition unit that acquires a first captured image captured by the imaging unit before moving to a target printing area of the liquid application device and a second captured image captured by the imaging unit after moving to the target printing area of the liquid application device; A template image is extracted from the first captured image, template matching using the template image is performed on the second captured image, and based on the position of an image portion in the second captured image that matches or is similar to the template image detected by the template matching, a relative deviation amount before and after the movement of the liquid application device is calculated. A second calculation unit; further comprising The liquid application device according to <2> or <3> above, wherein the correction unit performs second correction to move and rotate the printed image to be printed on the target printing area based on the deviation amount calculated by the second calculation unit. <6>The liquid application device according to <5> above, wherein the correction unit performs the first correction after performing the second correction on the printed image to be printed on the target printing area. <7>The liquid application device according to <5> above, wherein the second calculation unit extracts the template image from an image portion of the first captured image that overlaps the second captured image. <8>The liquid application device according to <5> above, further comprising a determination unit that determines the start position of the printing area in the second captured image determined by the template matching by the second calculation unit. <9>A liquid application system that divides print data into a plurality of print images and scans a discharge head to discharge liquid onto a printing surface based on each of the print images for a plurality of printing areas, a first acquisition unit that acquires distance information indicating the distance from a detection device that detects the distance from the printing surface; a first calculation unit that calculates the inclination of the plane scanned by the discharge head with respect to the printing surface from the distance information acquired by the first acquisition unit; a correction unit that corrects the print image based on the inclination calculated by the first calculation unit; A liquid application system having the above. <10>A liquid application device including the first acquisition unit, an external device including the first calculation unit and the correction unit; The liquid application system according to <9> above, including the above. <11>A liquid application method that divides print data into a plurality of print images and scans a discharge head to discharge liquid onto a printing surface based on each of the print images for a plurality of printing areas, an acquisition step of acquiring distance information indicating the distance from a detection device that detects the distance from the printing surface; a calculation step of calculating the inclination of the plane scanned by the discharge head with respect to the printing surface from the acquired distance information; a correction step of correcting the print image based on the calculated inclination; A liquid application method having the above. <12>On a computer included in a liquid application device that divides print data into a plurality of print images and scans a discharge head to discharge liquid onto a printing surface based on each of the print images for a plurality of printing areas, An acquisition step of acquiring distance information indicating the distance from a detection device that detects the distance to the printing surface; A calculation step of calculating the inclination of the plane scanned by the ejection head with respect to the printing surface from the acquired distance information; A correction step of correcting the printed image based on the calculated inclination; It is a program for executing the above.

Explanation of Signs

[0118] 1 Liquid application device 2 Carriage 2a Ink ejection head 3 Rail 4 Ink supply system 4a Pipe 5 Power supply system 6 Controller unit 7 Three-dimensional camera 8 Two-dimensional camera 9 GNSS receiver 10 Tire 11 Frame 12 Structure 13 Alignment member 14 LiDAR 15 Operation panel 21 Printing unit 22 Control unit 23 Head movement mechanism 24 Rail movement mechanism 30 External device 61 CPU 62 Memory 63 I / F 64 Unit control circuit 101 Imaging unit 102 Acquisition unit 103 Direction identification unit 104 Storage unit 105 Distance acquisition unit 110 Image processing unit 111 Displacement amount calculation unit 112 Inclination calculation unit 113 Printed image correction unit 114 Printing start position determination unit 120 Image formation unit 121 Movement control unit 122 Discharge control unit CAR1, CAR2, CAR_I, CAR_R Imaging areas CIM1, CIM2 Captured images Dx, Dy Displacement amounts Dθ Displacement angle IM Image PAR1, PAR2 Printing areas PAR_I, PAR_P, PAR_R Printing areas PIM1, PIM2, PIM2a, PIM11 Printed images PIM_A Corrected printed image PIM_B Printed image before correction RS Printing surface S Start position SP Step portion TP, TP1, TP11 Template images TP12, TP2 Comparison images

Prior art documents

Patent documents

[0119]

Patent Document 1

Claims

1. A liquid application device that divides print data into a plurality of print images and scans a discharge head to discharge liquid onto a print surface based on each of the print images for a plurality of print areas, comprising: a first acquisition unit that acquires distance information indicating the distance from a detection device that detects the distance to the print surface; a first calculation unit that calculates the inclination of the plane scanned by the discharge head with respect to the print surface from the distance information acquired by the first acquisition unit; a correction unit that corrects the print image based on the inclination calculated by the first calculation unit; A liquid application device comprising:

2. The liquid application device according to claim 1, wherein the correction unit performs a first correction to reduce distortion caused by the inclination based on the inclination calculated by the first calculation unit.

3. The liquid application device according to claim 2, wherein the correction unit calculates a skew amount when the print image is printed from the inclination calculated by the first calculation unit, and performs the first correction so as to reduce the skew amount of the print image.

4. The liquid application device according to any one of claims 1 to 3, wherein a detection range of the distance to the print surface by the detection device is larger than the print area.

5. A second acquisition unit that acquires a first captured image captured by an imaging unit before moving to a target print area of the liquid application device, and a second captured image captured by the imaging unit after moving to the target print area of the liquid application device; extracting a template image from the first captured image, performing template matching using the template image on the second captured image, and calculating a relative displacement amount before and after the movement of the liquid application device based on the position of an image portion in the second captured image that matches or is similar to the template image detected by the template matching; further comprising: The liquid application device according to claim 2 or 3, wherein the correction unit performs a second correction to move and rotate the print image to be printed on the target print area based on the displacement amount calculated by the second calculation unit.

6. The liquid application device according to claim 5, wherein the correction unit performs the first correction after performing the second correction on the print image to be printed on the target print area.

7. The liquid application device according to claim 5, wherein the second calculation unit extracts the template image from an image portion of the first captured image that overlaps with the second captured image.

8. The liquid application device according to claim 5, further comprising a determination unit that determines a start position of the printing area based on the printing area in the second captured image determined by the template matching by the second calculation unit.

9. A liquid application system that divides print data into a plurality of print images, scans a discharge head based on each of the print images for a plurality of printing areas, and discharges liquid from the discharge head onto a printing surface, a first acquisition unit that acquires distance information indicating the distance from a detection device that detects the distance from the printing surface; a first calculation unit that calculates an inclination of a plane scanned by the discharge head with respect to the printing surface from the distance information acquired by the first acquisition unit; a correction unit that corrects the print image based on the inclination calculated by the first calculation unit; A liquid application system having the above.

10. A liquid application device including the first acquisition unit, an external device including the first calculation unit and the correction unit; The liquid application system according to claim 9, including the above.

11. A liquid application method that divides print data into a plurality of print images, scans a discharge head based on each of the print images for a plurality of printing areas, and discharges liquid from the discharge head onto a printing surface, an acquisition step of acquiring distance information indicating the distance from a detection device that detects the distance from the printing surface; a calculation step of calculating an inclination of a plane scanned by the discharge head with respect to the printing surface from the acquired distance information; a correction step of correcting the print image based on the calculated inclination; A liquid application method having the above.

12. A program for causing a computer included in a liquid application device that divides print data into a plurality of print images, scans a discharge head based on each of the print images for a plurality of printing areas, and discharges liquid from the discharge head onto a printing surface to execute an acquisition step of acquiring distance information indicating the distance from a detection device that detects the distance from the printing surface; a calculation step of calculating an inclination of a plane scanned by the discharge head with respect to the printing surface from the acquired distance information; a correction step of correcting the print image based on the calculated inclination. ​

Images