Surface printer and line quality

Abstract

In an example, a surface marking robot may print a line to a surface. Images of the printed line may be captured. The captured image may be analyzed to calculate a parameter of the printed line. In response to determining a parameter of the printed line is not within limits, print settings may be modified. In one example, captured image may be analyzed to assess the health of the surface marking robot.

Description

SURFACE PRINTER AND LINE QUALITYBACKGROUND

[0001] Surface printers, including surface marking robots, may be used to draw or print lines on a surface by depositing printing material while moving.BRIEF DESCRIPTION OF THE DRAWINGS

[0002] Various non-limiting examples will be described with reference to the following accompanying drawings, in which:

[0003] FIG. 1 is a block diagram of a surface marking printer according to an example;

[0004] FIGS. 2A-2F are views illustrating a surface marking robot according to an example;

[0005] FIG. 3 illustrates a conceptual drawing of printed line according to an example;

[0006] FIGS. 4A-4B illustrate conceptual drawings of printed line according to an example;

[0007] FIG. 5 is a flowchart illustrating calibrating a surface marking printer for line analysis according to an example;

[0008] FIG. 6 illustrates a calibration pattern according to an example;

[0009] FIG. 7 illustrates a conceptual drawing of captured image of calibration pattern according to an example;

[0010] FIG. 8 illustrates printed test lines according to an example;

[0011] FIG. 9 is a flowchart illustrating a process for analyzing and modifying a printed line according to an example;

[0012] FIG. 10 illustrates a conceptual drawing of a captured image of printed line according to an example;

[0013] FIG. 11 illustrates a conceptual drawing of multiple image captures of printed line according to an example;

[0014] FIG. 12 illustrates a conceptual drawing of analysis of image captures of printed line according to an example;

[0015] FIG. 13 illustrates a conceptual drawing of multiple image captures of printed line according to an example;

[0016] FIG. 14 is a flowchart illustrating a process for analyzing and modifying a printed line according to an example;

[0017] FIG. 15 is a block diagram illustrating a computer program product according to an example;

[0018] FIG. 16 is a block diagram illustrating an example fluid delivery apparatus according to an example; and

[0019] FIG. 17 is a block diagram illustrating a hardware apparatus including a semiconductor package according to an example.DETAILED DESCRIPTION

[0020] Surface marking printers may include, for example, autonomous vehicles which may be used for printing images such as lines on surfaces (i.e. , a substrate) for applications such as construction and street marking. For example, an autonomous vehicle may be referred to as a surface marking robot and may receive a floorplan and print a layout for building construction on a floor. For example, a surface marking robot may print elements, such as, lines, text, dashed-lines, curved lines, circumferences, etc., which correspond to layouts of interior walls, mechanical, electrical, plumbing, fire protection, HVAC systems, and the like. In a typical application, it may be useful to utilize different line types during printing. For example, it may be useful to print a relatively wide line having a specific color for an interior wall layout and print a relatively narrow line having another specific color for an electrical layout. Further, a user may have specific preferences with respect to line properties and different types of applications may require different levels of precision with respect to line properties.

[0021] Typical flooring materials on which a layout may be printed may include porous surfaces, such as, for example, polished and rough concrete, tarmac, and wood and non-porous surfaces, such as, for example, terrazzo, vinyl, and epoxy. Further, the conditions of a particular flooring type may vary based on, for example, surface finishing, floor conditions (e.g., wet versus dry, shaded versus direct sunlight), and weather conditions (e.g., temperature and relative humidity).Additionally, printing fluids utilized for particular applications may vary as well. For example, a surface marking robot may utilize an ink type which may be permanent or semi-permanent and water-based or solvent based. Variabilities in substrates, environmental conditions, and printing fluids can result in differences in how a printing fluid is absorbed (e.g., more / less bleed, penetration, etc.) which results in differences in the appearance of printed lines.

[0022] Although a user may set a surface marking robot to print to a surface according to a specified line width, due to variabilities, for example, those described above, the resulting line may be wider or narrower than desired. Further, the quality and consistency of the resulting line may vary as well. For example, a line may appear to have a wavy edge versus a desired straight edge. That is, although a user may adjust the line width manually using provided predefined settings, in a typical case, in order to ensure that a printed line meets the user’s requirements, the user manually adjusts the predefined settings using trial-and-error until the printed line appears acceptable. Further, conditions (e.g., weather conditions) may be highly variable, and thus, the user may repeat the manual adjustment process numerous times during printing. This disclosure describes techniques that facilitate the printing of lines by a surface marking robot having a specified width and quality while reducing the need for manual adjustments as conditions change during a current job. Line width and line quality are examples of line properties that may be determined according to techniques herein.

[0023] Figure 1 is a block diagram of a surface printer according to an example. A surface printer may include a surface marking robot which may be self-propelled and apply a printing material on a surface. In some examples, a surface marking robot may be propelled with user assistance. In some examples, a surface marking robot comprises a motor, for example an electric motor, and a source of energy such as a battery. A surface marking robot may be propelled by friction, for example by a movement of wheels on the surface, or may be airlifted. Typical surfaces may include any of the example surfaces described above. For example, a surface may be a concrete, asphalt, or a wooden or a composite material surface. Further, in some examples, a surface may includefield or turf. A surface may comprise holes or obstacles. A surface marking robot may apply a printing fluid, for example, ink, to the surface while avoiding obstacles.

[0024] In some examples, the surface marking robot is operated by a computer system comprising a processor and a memory. In Figure 1 , example surface printer 100 includes processor 102, memory 104, motion control system 106, position detection system 108, printing control system 110, printing system 112, sensors 114, user interface 116, calibration system 118, and communications system 120. It should be noted that although example surface printer 100 is illustrated as having distinct functional blocks, such an illustration is for descriptive purposes and does not limit surface printer 100 to a particular hardware or machine readable instruction architecture. Functions of surface printer 100 may be realized using any combination of hardware, and / or machine readable instruction implementations. In one example, functions of surface printer 100 may include various chipsets connected via a system interface. For example, a system interface may include a chipset supporting PCI and PCIe bus protocols, proprietary bus protocols, Universal Serial Bus (USB) protocols, I2C, or a system interface may include any other logical and physical structure that may be used to interconnect peer devices.

[0025] Processor 102 may implement functionality and / or process instructions for execution in surface printer 100. Processor 102 may include processing units(s) capable of retrieving and processing instructions, code, and / or data structures for implementing techniques described herein. Instructions may be stored on a computer readable medium, such as memory 104 or internal or external storage devices. Processor 102 may include digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Processor 102 may include multi-core central processing units.

[0026] Memory 104 is may store information that may be used by surface printer 100 during operation. Memory 104 may be described as a non-transitory or tangible computer-readable storage medium. Memory 104 may include anytype of memory device or storage medium capable of storing data. A storage medium may include tangible or non-transitory computer-readable media. Computer readable media may include optical discs, flash memory, magnetic memory, or any other suitable digital storage media. In some examples, a memory device or portions thereof may be described as non-volatile memory and in other examples portions of memory devices may be described as volatile memory. Examples of volatile memories may include random access memories (RAM), dynamic random access memories (DRAM), and static random access memories (SRAM). Examples of non-volatile memories may include magnetic hard discs, optical discs, floppy discs, flash memories, or forms of electrically programmable memories (EPROM) or electrically erasable and programmable (EEPROM) memories. A device may store instructions for in a suitable, non- transitory computer-readable medium and execute the instructions in hardware using processors.

[0027] Motion control system 106 may propel surface printer 100. In some examples, motion control system 106 may comprise wheels driven by a motor (e.g., an electric motor), or any suitable propulsion system. In some examples, motion control system 106 may comprise control circuitry to control a motor to drive wheels to control a direction and speed of surface printer 100. In some examples, motion control system 106 may include a microcontroller following a trajectory servo in communication with a propulsion system comprising motor driver electronics to supply force to a set of wheels. In some examples, motion control system 106 may also comprise a processor to receive and execute instructions defining an intended path or trajectory for surface printer 100 to follow.

[0028] Position detection system 108 may enable a position of surface printer 100 to be detected. For example, position detection system 108 may receive a guiding system signal, which may be used to detect a position of surface printer 100. For example, surface printer 100 may be guided using a remote guiding system. A guiding system may permit guiding the surface printer from a reference point corresponding to the location of the guiding system. A guiding system may communicate guiding system position information with the surface printer usingelectromagnetic waves or radiation. Guiding system position information may provide the location of the guiding system, such that the guiding system may be used as a reference point (or beacon) to calculate or estimate an absolute position of surface printer 100. Example guiding systems can include a Wi-Fi access point, an ultrasound beacon, a total station, a laser tracker or an interferometer. In some examples, a guiding system may be an optical guiding system. In this case, the communication between the guiding system and surface printer 100 may take place, for example, through visible or infrared light. For example, position detection system 108 may include a reflector to reflect a measuring beam to a source. In some examples, a guiding system may include a Global Navigation Satellite System (GNSS) receiver providing an absolute position of the guiding system.

[0029] Position detection system 108 may for example, receive information from sensors 114. Sensors 114 may include sensors for use with position detection system 108, as well as addition sensors described in detail below. Sensors for use with position detection system 108 may include any kind of suitable position sensor, such as, for example, rotary encoders located on wheels of surface marking robot 100, a camera located on the body of surface printer 100, a Light Detection and Ranging (LIDAR) system, an inertial mechanical unit to sense accelerations and direction, a combination including at least some of the previously-mentioned position sensors, or any other suitable kind of position sensor. In some examples, position detection system 108 may compare information from sensors 114 with a servo path to detect deviations. For example, accelerations in an axis other than that defined by the servo path can indicate that surface printer 100 is not following the defined servo path. In some examples, a determination that rotary encoders on the wheels are not increasing steadily can provide an indication that surface printer 100 has deviated from the defined path.

[0030] In some examples, position detection system 108 and / or motion control system 106 may calculate a magnitude and direction of the difference between the current position of surface printer 100 and an intended path and may correct the path accordingly. That is, motion control system 106 and position detection system 108 may operate in conjunction to cause surface printer 100 totraverse an intended path. In some examples, position detection system 108 may comprise processing circuitry to calculate whether a detected position matches an intended path and cause motion control system 106 to make adjustments.

[0031] As described above, a surface marking robot may receive a floorplan and print a layout for building construction on a surface. Printing control system 110 and printing system 112 may cause a representation of an image to be printed by surface printer 100. For example, in one example, printing control system 110 may receive printjob commands and / or data corresponding to a print job (e.g., image data) and generate print data such that a print job is executed. In some examples, printing control system 110 may reproduce print data from received data. In some examples, the received data itself may already correspond to print data. In other examples, print data may be dynamically generated during printing of a print job. Further, print data may also be stored from the outset in a memory and printing control system 110 may then dynamically access the print data during execution of a print job.

[0032] Printing system 112 may cause printing material to be deposited. For example, printing system 112 may include a printhead carriage including a plurality printheads, where a printhead comprises a die forming a plurality of nozzles. The nozzles may be aligned in columns along a length of a printhead. For example, a printhead carriage may comprise a plurality of ink-jet printheads. A printing fluid, including, for example, ink or a modelling agent, may be ejected through the nozzles of the printhead. In this manner, printheads included in printing system 112 may deposit ink onto a surface, thereby printing an image corresponding to a print job. It should be noted that in other examples, printing system 112 may include a thermal or piezo-electric printhead. Further, it should be noted that ink is used herein as an example, and in other examples, other printing fluids, such as, pre-printing (e.g., cleaning fluid) and post-printing agents (e.g. varnishes, glosses, under-treatments) may alternatively be deposited. Printing system 112 may cause printing material to be deposited according to settings. As described in further detail below, in some examples, a print system may include the following firing settings: aperture time, ink pressure, and dropspacing, which may be adjustable. In other examples, firing settings may include energy, voltage, pulse width, etc.

[0033] As described above, in some examples, printing control system 110 may receive data corresponding to a print job. In some examples, data corresponding to a print job may correspond to a floorplan. A floorplan may include a two dimensional or three dimensional representation of a structure such as, for example, a building. In some examples, a floorplan comprises floorplan features corresponding to objects or characteristics, such as, for example, walls, windows, doors, staircases, elevator cases, sinks, types of finish, construction methods, materials, symbols for electrical, mechanical, plumbing, fire protection, HVAC systems, gas, or water supply features, etc. In some examples, a floorplan comprises features corresponding to traffic, parking or road surface marking, aimed at pedestrians or vehicles.

[0034] In one example, a digital data file may comprise digital data associated with a floorplan. For example, an input file, or a raw input file, may include a digital representation of a drawing provided by a user, for example in a DXF (Drawing Exchange Format), DWG (DraWinG) or BIM (Building Information Modeling) format. In one example, an input file may be parsed in layers by surface printer 100, for example, for checking that relevant printing information is contained in the file. In one example, an input file may be processed by surface printer 100 to detect obstacles which may impact a trajectory or path to be followed by surface printer 100. In one example, an input file may be processed by surface printer 100 to sort and group clusters of graphical representations such as lines or text. In one example, an input file may be processed by surface printer 100 for path planning, for example in order to sort an order in which graphical representation elements such as floorplan features may be printed, while avoiding obstacles and reducing a printing time. In one example, an input file may be processed by surface printer 100 for printing or marking for example by printing control system 110.

[0035] Communications system 120 may enable surface printer 100 to communicate with external computing devices via networks. For example, communications system 120 may enable surface printer 100 to communicate withother computing devices connected to local area network and / or wide area network. Communications system 120 may be included as part of a network interface card and may include an optical transceiver, a radio frequency transceiver, or any other type of device that can send and receive information. Communications system 120 may operate according to communications protocols including for example, Global System Mobile Communications (GSM) standards, code division multiple access (CDMA) standards, 3rd Generation Partnership Project (3GPP) standards, European Telecommunications Standards Institute (ETSI) standards, Internet Protocol (IP) standards, Wireless Application Protocol (WAP) standards, and IEEE standards, such as, for example, IEEE 802 standards (e.g., Wireless LAN, PAN, etc.). In one example, communications system 120 may enable surface printer 100 to receive an input file from an external computing device (e.g., a laptop computer or the like).

[0036] As described above, in some examples, surface printer 100 may be guided using a remote guiding system. In some examples, surface printer 100 may additionally be controlled using input received from a user via an external computing device in communication with surface printer 100. For example, input for controlling surface printer 100 may be generated from an input device, such as, for example, a touch-sensitive screen, a track pad, a joystick, a mouse, or a keyboard. Input may include for example, at least, input to control the movement of surface printer 100, input to modify printing settings, input to modify line properties, and input to modify additional controllable settings (e.g., maximum, minimum speed, etc.) User interface 116 may include an input device and / or a basic user interface. For example, in one example, user interface 116 may include basic manual controls (e.g., Power ON / OFF buttons and / or switch) and basic status indicators (e.g., LED lights indicating a power and / or operating states).

[0037] As described above, sensors 114 may include sensors for use with position detection system 108. Sensors 114 may further include collision avoidance and safety sensors. For example, LiDAR sensors may be used for collision avoidance and safety sensors may be used to detect cliffs or ledges to prevent surface printer 100 from falling during operation. As described above,environmental conditions may vary during operation. Sensors 114 may additionally include sensors to measure environmental conditions, for example, temperate sensors, humidity sensors, and optical sensors for measuring an amount sunlight shining on a surface. Sensors 114 may further include image capturing sensors, for example, digital cameras. Calibration system 118 may provide calibration for a printing process. For example, calibration system 118 may cause settings of printing system 112 to be adjusted based on, for example, surface properties, print material properties, environmental conditions, and the like. As described in further detail below, calibration system 118 may further analyze images captured from an image capturing sensor and cause settings of printing system 112 to be adjusted.

[0038] As described above, a surface printer may include a surface marking robot which includes a printing system to cause printing material to be deposited to a surface while the surface marking robot traverses an intended path. Figures 2A-2F are views illustrating a surface marking robot according to an example. It should be noted that Figures 2A-2F illustrate exterior views of an example surface marking robot 200 and elements illustrated in Figures 2A-2F may operate in conjunction with elements described above with respect to Figure 1. For example, in an example, surface marking robot 200 may include a processor, a memory and other systems illustrated in Figure 1 . As illustrated in Figures 2A- 2F, surface marking robot 200 includes drive wheels 202A-202B, caster wheel 204, user interface 206, reflector 208, printhead 210, image capture camera 212, LiDAR sensors 214A-214C, safety sensors 216A-216B, and position sensing camera 218. As described above, a surface printer may include a motion control system to propel the surface marking robot. Drive wheels 202A-202B are wheels driven by a motor and caster wheel 204 rotates such that surface marking robot 200 is able to move about a direction and speed. In one example, surface marking robot may have a maximum navigation speed of 1 ,440m / h and a maximum printing speed of 900m / h. As described above, a surface printer may include basic manual controls and basic status indicators. User interface 206 provides a basic user interface for surface marking robot 200. For example, user interface 206 may include an ON / OFF button.

[0039] As described above, a surface printer may include a position detection system including a reflector to reflect a measuring beam to a source. Reflector 208 may reflect a measuring beam to a source. In one example, reflector 208 includes a 360 degree prism. As described above, a surface printer may include sensors for use with a position detection system. LiDAR sensors 214A-214C and position sensing camera 218 may be used with a position detection system. As further described above, a surface printer may include sensors for collision avoidance and safety sensors. Safety sensors 216A-216B include sensors for detecting cliffs or ledges to prevent surface marking robot 200 from falling over a ledge during operation. In one example, if safety sensors 216A-216B indicate surface marking robot 200 is approaching a ledge, motion of surface marking robot 200 is ceased.

[0040] As described above, a surface printer may include a printing system including a printhead carriage including a plurality printheads, where a printhead comprises a die forming a plurality of nozzles. Printhead 210 represents an example of a printhead having nozzles to deposit a printing material to a surface. As described above, a surface printer may print a line having a specified width. For example, in one example, printhead 210 may print lines having a specified width in the range of 2mm to 51 mm with an accuracy tolerance of 3mm. Further, as described above, surface marking robot 200 may have a maximum printing speed of 900m / h.

[0041] As illustrated in Figure 2B, the distance between printhead and a surface may be represented as distance d. Further, distance d corresponds to the distance between image capture camera 212 and a surface. In one example, d may be approximately 19mm. It should be noted that d may vary during the lifetime of surface marking robot 200. For example, as drive wheels 202A-202B wear due to use over time, d may decrease. As described in further detail below, image capture camera 212 may capture images of a calibration pattern and printed lines to accommodate for changes in d. In some examples, image capture camera 212 may include a relatively low-cost and / or relative low resolution digital camera. For example, image capture camera 212 may include a video graphics array (VGA) camera having a 640x480 pixel resolution. In other examples, imagecapture camera 212 may include a 1 , 1 .3, 2, 3, 5, etc., megapixel camera. Image capture camera 212 may include a light for illuminating a surface and to serve as a flash during image capture. Further, it should be noted that image capture camera 212 may be synchronized with the droplet firing, e.g., images may be captured after a droplet is fired or within a predetermined time after a droplet is fired.

[0042] As described above, although a user may set a surface marking robot to print to a surface according to a specified line width, due to variabilities the resulting line may vary from an ideal or fitted line. Figure 3 illustrates a conceptual drawing of printed line versus a fitted line. As illustrated in Figure 3, the appearance of the line is less than ideal. That is, as illustrated in Figure 3, although an ideal line may be a uniform line of width W, a line that is actually printed may deviate from an ideal line and have a width that is variable about its length. It should be noted that, as described in further detail below, because a width of a line may be variable about a length, line width may be defined as a width of a line at a point along a length and / or according to functions, e.g., an average of widths at points about the length of a line. Further, it should be noted that line quality may be evaluated based on properties such as, subjective appearance or evaluated based on properties which include objectively defined line quality parameters. ISO / IEC 13660:2001 , Information technology — Office equipment — Measurement of image quality attributes for hardcopy output — Monochrome text and graphic images, September, 2001 provides several defined line quality parameters for evaluating the quality for a line. For example, line quality parameters described in ISO / IEC 13660:2001 include: line width which may be defined as the average line width in relation to a theoretically defined line width, line roughness which may be defined as the variability of line based on a standard deviation of residuals from a fitted line, and line sharpness, which may be defined as a cross-section profile of a line edge describing characteristics of transition from black to white. As described above, line properties, including, for example, defined line quality parameters, and / or additional or alternative line parameters may be determined according to techniques herein. For example, according to the techniques herein, other line properties, for example, linesharpness, line contrast, printing material density, etc. may be determined and evaluated.

[0043] As described above, a printing system of a surface marking robot may cause ink to be ejected through the nozzles of a printhead. That is, a printing system may cause drops of ink to be deposited to a surface. Figures 4A-4B illustrate drops of ink corresponding to the printed line illustrated in Figure 3. Figure 4A illustrates drops, Do-De, deposited to a surface at an initial time. It should be noted that for some applications, relatively uniform drops forming a line may be considered acceptable. That is, for some applications, a rectangular line having straight edges may not be desired. Thus, for some applications, drops Do-De may be considered acceptable. That is, circles with a uniform diameters that fit a line may be acceptable for some applications.

[0044] Figure 4B illustrates variation, or deviations from ideal drops, of the drops, Do-De, after an amount of time. That is, as illustrated in Figures 4A-4B, based on the properties of a surface and corresponding environmental conditions, the absorption of drops on a surface may change after being initially deposited and not be absorbed in a manner that forms an acceptable line. That is, for example, depending on surface and environmental conditions, ink penetration may vary and spreading or bleeding may occur. For example, in the case where a surface is concrete, for various concrete finishings, lines could have visible differences that could impact the line width and quality. As described above, in a typical case, in order to ensure that a printed line meets a user’s requirements, a user would manually adjust predefined settings, print a line, evaluate the subjective appearance of a line, and repeat this process using trial-and-error until the printed line appears acceptable. It should be noted that for a typical application, a surface marking robot may perform jobs over numerous hours and typical processes for ensuring a line meets a user’s requirements could delay completion of a job by at least several hours.

[0045] This disclosure describes techniques that facilitate the printing of lines having a specified properties, including, for example, width and quality, by a surface marking robot, while reducing the need for manual adjustments. That is, according to the techniques herein, line properties may be adjusted by a surfacemarking robot while printing. In one example, according to the techniques herein, a surface marking robot may make closed loop corrections to a line with minimal user intervention. For example, as described in further detail below, an image capture camera and a calibration system may capture and analyze images of a line and cause printer settings to be adjusted. That is, according to the techniques herein, a surface marking robot is enabled to maintain a selected line width as conditions change, by monitoring a line in real-time and adjusting printing settings accordingly, in some cases, with minimal supervision of a user. Compared to typical processes, processes enabled according to the techniques described herein may be more consistent and less prone to subjective errors and reduce delays. Further, it should be noted that unacceptable line properties may be due to an operational issue with a printing system. That is, a nozzle may not be firing correctly. Thus, the techniques described herein may be used to detect if a printing system is operating correctly, for example, by monitoring nozzle health.

[0046] The techniques described herein may be generally applicable, regardless of the printing material used, the surface for application, and the weather conditions, which may change during the current job. Further, the techniques described herein may allow for a surface marking robot to generate and have access to a media-properties dictionary, which may be used to replicate printing settings in similarly detected conditions. Further, a surface marking robot may create profiles or modify existing profiles and profiles may be communicated to other surface marking robots.

[0047] As described above, the distance between a printhead and image capture sensor and a surface may vary during the lifetime of surface marking robot. The variation may be due to wear, repair, and / or replacement of drive wheels. Further, the distance may vary for each manufactured surface marking robot due to mechanical tolerances. As further described above, a calibration system of a surface printer may analyze images captured from an image capturing sensor. According to the techniques herein, a calibration process may be performed, for example, by a calibration system, in order to ensure line properties can be accurately determined from captured images.

[0048] In one example, a calibration process may be performed in order to calculate a pixel to distance ratio. For example, as described above, distance d may be 19mm, but, d may vary during the lifetime of a surface marking robot (e.g., + / -2mm). As further described above, an image capturing camera may have a resolution of 1 Megapixel or the like. A variation in d may cause corresponding image capture camera 212 to zoom in / out on a surface and change the area of the surface included in a captured image. Thus, a calibration process may be used to calculate a pixel / mm ratio of a captured image. Figure 5 is a flowchart illustrating calibrating a surface marking printer for line analysis according to an example. Figure 6 illustrates a calibration pattern according to an example. Figure 7 illustrates a conceptual drawing of captured image of calibration pattern according to an example.

[0049] As illustrated in Figure 6, a calibration pattern may include multiple lines each having a specified width (e.g., 1.0 mm, 1.5 mm, and 2.0 mm). As illustrated in Figure 7, a captured image of a calibration pattern includes the multiple lines captured within an image resolution (e.g., 720x720 pixels). It should be noted that a captured image may include a captured image that has undergone image processing. For example, the image illustrated in Figure 7 may correspond to a cropped 1 Megapixel image. Further, image processing may include reducing or eliminating any noise in the captured image. Additionally, an image captured by an image capture camera may be in color and image processing may be used to convert the actual captured image to a black and white image. According to a calibration process, a pixel to distance ratio (e.g., pixel / mm ratio) of lines in a captured image may be determined.

[0050] Referring to Figure 5, at 502 of calibration process 500, a calibration pattern, for example, calibration pattern illustrated in Figure 6, is aligned at a position in the view of an image capture camera of a surface marking robot, for example, image capture camera 212. For example, a calibration pattern may be printed to a sheet of paper and the sheet of paper may be placed under surface marking robot 200 at an appropriate position. For example, in addition to the calibration pattern, reference markers for aligning the calibration may be printed to the sheet of paper (e.g., markers corresponding to wheel positions). At 504,an image of the calibration pattern is captured. For example, surface marking robot 200 may cause image capture camera 212 to illuminate and capture an image of the calibration pattern, i.e. , take a picture of the calibration pattern. At 506, the captured image may be analyzed. For example, by a calibration system. As described above, a captured image may undergo image processing. Image processing may be included as part of the image analysis. For example, pixels at the edge of a line may appear as grey rather than as a distinct black or white pixel. In this case, the image may be processed to classify each pixel as black or white.

[0051] Referring again to Figure 7, for each of the three lines, a number of pixels corresponding to the width of the line is provided, 78 pixels for the 1 .0mm line, 117 pixels for the 1.5mm line, and 152 pixels for the 2.0mm line. That is, image analysis at 506 may calculate a number of pixels corresponding to the width of each line. For example, image analysis may include: converting an image to a black and white image, such that each pixel is stored as a black or white intensity value, selecting a position about the x-axis of the captured image (e.g., position 360), for each pixel about the y-axis at position x (e.g., (360, 0) to (360, 719)), determining the intensity value of each pixel, and counting a number of consecutive pixels of each intensity. In some examples, the process may include creating an array of pixels and finding peaks in the array.

[0052] Referring to Figure 5, at 508, a pixel to distance ratio is calculated. With respect to Figure 7, Table 1 provides the actual width of each line of the calibration pattern and the measured pixel width as determined from the image analysis. Further, in Table 1 , a ratio is provided as Actual width divided by Pixel width measured.Table 1

[0053] The values in Table 1 may be used to calculate a pixel to mm ratio. For example, the average of the Ratio values in Table 1 is 0.0129 mm / px. In one example, this may be used as the calculated pixel to mm ratio. In other examples, the calculated pixel to mm ratio may be set to the minimum value (i.e., 0.0128), the maximum value (i.e., 0.0132), or may be set to a value based on a function. For example, depending on an application and / or precision, an average of the minimum and the maximum pixel to mm ratio may be used to calculate a pixel to mm. It should be noted, in the image analysis process described above, a position about the x-axis may be selected. For a uniform line in a calibration pattern, selecting a single position about the x-axis may be acceptable. However, in some examples, multiple positions about the x-axis may be sampled in order to calculate pixel to distance ratio. For example, a captured image may be sampled at every N (e.g., 10) positions and the number of counted pixels at each of the N positions may be averaged. Further, it should be noted that with respect to the values provided in Table 1 , these values are example values for illustrative purposes. That is, for a particular implementation, field of view, and thus, a pixel distance ratio, will depend on advance speed (e.g., firing frequency), camera resolution, and focus (e.g., distance). The techniques described herein may be generally applicable to various surface printers and image capture sensor implementations.

[0054] Finally, as illustrated in Figure 5, at 510, the calculated pixel to distance ratio is stored. For example, the calculated pixel to mm may be stored to a memory of surface marking robot 200 and / or to a memory of an external computing device. It should be noted that in other examples, a calibration process may include determining and storing additional information. For example, in one example, a calibration process may establish if the variation of the ratio of each of the lines is greater than a threshold and provide an indication of confidence with respect to a calculated pixel to distance ratio. For example, an image capture camera and printhead may be damaged and / or misaligned and an indication of low confidence may be stored and further used to diagnose a operational issue with a surface printing robot.

[0055] As described above, a calibration process may be performed in order to ensure line properties can be accurately determined from captured images. As further described above, input to a surface marking robot may include user specified line properties. For example, surface marking robot 200 may receive a user selected line width from an external computing device in communication with surface marking robot 200. For example, a user may select a line width from a set of selectable line widths (e.g., 1 mm line widths in the range of 2mm to 51 mm) using a graphical user interface displayed on a portable computing device and the selected width may be communicated to surface marking robot 200. In response to receiving the selected line width, settings of a print system may be set to values corresponding to the selected line width. For example, a print system may include the following firing settings: aperture time, ink pressure, and drop spacing, which may be set according to the selected line width.

[0056] In one example, surface marking robot 200 may print a set of sample lines to a surface. For example, for a selected line width, multiple lines with various combinations of printing and / or firing settings corresponding to the selected line width may be printed. For example, each of the lines in a set may have the same settings for aperture time, drop spacing, and printing speed, but ink pressure may vary for each of the lines. Figure 8 illustrates printed test lines corresponding to an example. In the example illustrated in Figure 8, each of lines 0 to 4 appear to be similar in width, i.e., each may correspond to a selected width. However, as illustrated, the appearance of the lines, as printed, varies. In one example, according to the techniques described in further detail below, captured images of the printed test lines may be analyzed and one of the printed test lines and corresponding printing settings may be selected. That is, one of lines 0 to 4 may be selected as a line matching a selected line width, in order to select initial printing settings. In one example, surface marking robot 200 may print a set of test lines to a surface and a user may select a line from the set of printed lines. For example, referring to Figure 8, a user may input one of 0 to 4 to indicate an acceptable line appearance, which is communicated to surface marking robot 200.

[0057] According to techniques herein, in order to ensure that printed lines satisfy a user’s specified line properties and / or acceptable line appearance, images of printed lines may be analyzed. Figure 9 is a flowchart illustrating a process for analyzing and modifying a setting used to print a line according to an example. It should be noted that process 900 may be utilized in preparation for a printjob and / or during the completion of a print job. For example, lines analyzed using process 900 may include test lines and process 900 may be used to selecte an initial set of printing settings. Further, a line analyzed with process 900 may include a line printed during a print job and process 900 may be used to adjust printing settings accordingly to ensure printed lines are satisfactory. That is, process 900 may be used as a closed loop process while printing to ensure consistent line quality during a job.

[0058] Referring to Figure 9, at 902, a line width is received. For example, as described above, surface marking robot 200 may receive a user selected line width. Additionally, as described above, in some examples, surface marking robot 200 may receive a user selection corresponding to a printed test line. For purposes of process 900, a user selection corresponding to a printed test line, (i.e. , a user selected line preference) may be considered an aspect of a received line width. As described above, a surface marking robot may include printing settings. As further described above, surface marking robot may include sensors to measure environmental conditions, for example, temperate sensors, humidity sensors, and optical sensors for measuring an amount of sunlight shining on a surface. At 904, printing settings and environmental conditions corresponding to a selected line width may be determined. For example, aperture time, drop spacing, and ink pressure, as well as temperature and relative humidity may be determined.

[0059] As described above, a surface marking robot may generate and have access to a media-properties dictionary, which may be used to replicate printing settings in similarly detected conditions. In one example, based on a selected line width, a detected temperature, relative humidity and / or other detected or received conditions, surface marking robot 200 (or an external computing device) may access a media-properties dictionary in order to lookup printing settings, forexample, aperture time, drop spacing, and ink pressure, in order to produce the selected line width. Further, in some examples, similar to the selection of line width described above, a user may be able to select a surface type from a set of selectable types and / or input conditions. As described in further detail below, a media-properties dictionary may be updated.

[0060] Referring to Figure 9, at 906, a line is printed. As described above, the printed line may be a test line or a line printed during a job. At 908, surface marking robot 200 captures image(s) of the printed line. A process of capturing a printed line is similar to the process of capturing a calibration image, as described above. That is, for example, surface marking robot 200 may cause image capture camera 212 to illuminate and capture an image of the printed line. At 910, the captured image of the line is analyzed and the width of the line is calculated.

[0061] Figure 10 illustrates a conceptual drawing of a captured image of printed line according to an example. As described above, during a calibration process a calculated pixel to distance ratio is stored. This calculated pixel to distance ratio may be used to calculate a width of a line. For example, as described above, image analysis of a calibration pattern may include: converting an image to a black and white image, selecting a position about the x-axis of the captured image, for each pixel about the y-axis at position x, determining the intensity value of each pixel, and counting a number of consecutive pixels of each intensity. During calibration, the number of pixels and known widths of lines of the calibration may be used to calculate a pixel to distance ratio. Referring to Figure 10, in a similar manner, to the image analysis of a calibration pattern, a position about the x-axis may be selected and a number of consecutive pixels of each intensity may be counted. For the intensity corresponding to deposited printing material, the counted number of pixels may be multiplied by the stored pixel to distance ratio to calculate the width of the printed line. For example, if the stored pixel to mm ratio is 0.0129 mm / px and the number of counted pixels is 216, the width of the printed line may be calculated as 2.79mm.

[0062] As illustrated in Figure 10, because the printed line is illustrated as discrete drops, the width of the line varies along the length of the line. Asdescribed above, with respect to Figures 4A-4B, for some applications, relatively uniform drops forming a line may be considered acceptable. Thus, drops Do to Ds may form an acceptable line having a width that is equal to the diameter of drops Do to Ds. In the image analysis process described above, a position about the x-axis may be selected or a capture image may be sampled about the x-axis. With respect to Figure 10, multiple positions about the x-axis may be sampled in order to calculate a width the line. For example, in one example, y-axis pixels may be counted for every x position, and the full shape of the line may be determined. In one example, a width of the line may be calculated as the average of a set of widths at N sampling positions. For example, at every 10 horizonal samples, a width may be determined and the determined widths may be averaged to calculate a width for the printed line.

[0063] As described above, image capture camera 212 may be synchronized with the droplet firing such that images may be captured after a droplet is fired or within a predetermined time after a droplet is fired. As further described above, with respect Figures 4A-4B, the absorption of drops on a surface may change after being initially deposited and may not be absorbed in a manner that forms an acceptable line. Thus, the appearance of drops Do to Ds in Figure 10 may change after an image is captured. In one example, according to the techniques herein, the width of a line may be calculated based on multiple captured images. For example, the width of a line may be calculated based on multiple overlapping captured images. Figure 11 illustrates a conceptual drawing of multiple image captures of printed line according to an example. In the example illustrated in Figure 11 , Imageo corresponds to the captured image in Figure 10, i.e., an image captured an initial time, and Imagei corresponds to an additional captured image at a second time with an overlapping region. Figure 12 further illustrates Imageo and Imagei.

[0064] As illustrated in Figure 12, both Imageo and Imagei include drop D4, however, the appearance of D4 differs between images. That is, as described above, after a printing material is deposited, spreading or bleeding may occur. In the example illustrated in Figure 12, in Imageo, D4 is captured after initially being deposited and in Imagei, D4 is captured at a later time, after spreading hasoccurred. Thus, Samplei in Imageo is at an x position that may be used to evaluate an initial diameter of D4 and Sample2 in Imagei is at an x position that may be used to evaluate a diameter of D4 after absorption.

[0065] As illustrated in Figure 12, Imageo includes Sampleo and Samplei and Imagei includes Sample2 and Samples, where Sampleo and Sample2 are illustrated at the same x position (i.e. , at a left offset in the images) and Samplei and Samples are illustrated at the same x position (i.e., at a right offset in the images). Each of Sampleo to Samples may be used to evaluate a line width and / or a line quality. Each of Sampleo and Sample2 may represent drops after an initial absorption period and the widths at these positions may be useful for determining a width of a line. In one example, the width of the line may be calculated as the width of one of Sampleo and Sample2. In one example, the width of the line may be calculated as the average of Sampleo and Sample2.

[0066] Referring again to Figure 9, at 912, a determination is made whether width difference of samples is greater than a threshold. That is, for example, referring to Figure 12, the difference between Samplei and Sample2 may indicate the spreading of D4 after initially being deposited. Thus, if the difference between Samplei and Sample2 is greater than a threshold, this could indicate excess spreading of a printing material, which could indicate unacceptable line quality. Further, each of Samplei and Samples represent diameters of drops after initially being deposited and in some cases may be expected to be the same. Thus, if the difference of the widths of Samplei and Samples is greater than a threshold, this could additionally indicate unacceptable line quality. It should be noted that a difference in the widths of Samplei and Samples could indicate degradation in the performance of nozzles.

[0067] As illustrated in Figure 9, at 914, after it is determined that a difference is greater than a threshold, which may indicate unacceptable line quality, a printing process is stopped and a warning is provided to a user. That is, for example, surface marking robot stops the printing process and causes an indication to be provided to a user. For example, an indication could alert a user if surface marking robot is not printing and warn the user about the magnitude ofthe operational issue, so that the user may make decisions like stopping the job and servicing the surface marking robot.

[0068] At 916, based on the condition in 912 not being satisfied, a determination is made whether a line width is within acceptable limits. That is, for example, a specified line with may be 10mm and the range of acceptable limits may be within + / -2mm. Thus, in this example, if the width of the line calculated at 910 is between 8mm and 12mm, the line width may be considered within limits. As illustrated in Figure 9, at 918, after it is determined that the line width is considered within limits, settings and conditions are stored. For example, as described above, a surface marking robot may generate and have access to a media-properties dictionary. Storing settings and conditions allows a mediaproperties dictionary to be updated.

[0069] At 920, based on the condition in 916 not being satisfied, a determination is made whether the line is wider than a target. That is, for example, for the example described above, where the range of the limit is between 8mm and 12mm, a line not being within these limits is either less than 8mm or greater than 12mm. In this example, the determination at 920 would evaluate if the line width is greater than 12mm and in this example, if the line is greater than 12mm, at 922, settings are adjusted to produce smaller drops. That is, for example, for a surface marking including at least aperture time, ink pressure, and drop spacing firing settings, aperture time and / or ink pressure may be reduced. If the determination at 920, is not satisfied, this would indicate a drop is smaller than the lower end of a limit range (e.g., less than 8mm), in this case, as illustrated at 924, settings are adjusted to produce larger drops. For example, surface marking robot 200 may increase aperture time.

[0070] It should be noted that each of 908 to 924 may occur as a surface printer continues printing or the printing process may be paused. For example, if is determined at 916, that a line width is not within limits, printing may be paused (i.e., a surface marking robot stops moving and stops depositing material) until 922 or 924 is completed. For example, if a variation of line is detected, due to weather conditions, changes on the surface, or any other reason, depending on the variation, a surface printer could stop or slow down printing and makechanges in the firing settings in order to minimize the length of the job being affected. Once it is determined that a line is back to being acceptable, the surface printer could resume printing at the normal speed.

[0071] As illustrated in Figure 9, each of 906 to 924 may be completed as part of a closed loop. That is, images may be analyzed and adjustments may be continuously made, as needed. It should be noted that process 900 may be repeatedly initiated during a print job. For example, the process may repeat at regular intervals after 918. As described above, a surface marking robot may have a maximum printing speed of 900m / h. Thus, a printed line may extend at least over several meters. In some examples, process 900 may be repeated at regular time or distance intervals. For example, the process could be repeated every minute, which would correspond to every 15m at a maximum printing speed of 900m / h. As described above, with respect to Figures 11 and 12, in some examples, process 900 may include capturing overlapping images. Figure 13 illustrates a conceptual drawing of a line where multiple overlapping images are captured along the distance of the line. Each of the overlapping images in Figure 13 may correspond to repeating process 900 at regular intervals.

[0072] As described above, in addition to line width, according to the techniques herein, other line properties or parameters may be determined and evaluated. Figure 14 is a flowchart illustrating a process for analyzing and modifying settings used to print a line according to an example. Referring to Figure 14, at 1402, a line parameter is received, e.g., a user selected line parameter. At 1404, printing settings and conditions corresponding to a line parameter may be determined. At 1406, a line is printed. At 1408, image(s) of the printed line are captured. At 1410, the captured image of the line is analyzed and a line parameter is calculated. At 1412, a determination is made whether a printed error is detected, for example, based on a width difference of samples being greater than a threshold. At 1414, after it is determined that a printing error is detected, a printing process is stopped and a warning is provided to a user. At 1416, based on a printing error not being detected, a determination is made whether a line parameter is within acceptable limits. At 1418, after it is determined that the line parameter is within limits, settings and conditions are stored. At 1420,after it is determined that the line parameter is not within limits, settings are adjusted in order to produce a line having line parameters with limits and 1406 may be repeated.

[0073] As described above, in the case where a surface is concrete, for various concrete finishings, lines could have visible differences that could impact the line width and quality. Further, a concrete surface may have zones with or without direct exposition to the sun, showing temperature variation, or with different humidity. It should be noted that tests with different concrete formulations at different environmental conditions show that drop sizes may change as much as 65% to form an acceptable line and the relationship between environmental condition changes and changes to a drop size are not linear or easily predictable. Further, some internal flooring is made of various decorative materials, such as plywood, PVC panels or sheets, tiles, etc. These materials may also produce significant changes to drop shape and size based on conditions. Processes described above may be used to adapt drop sizes in various conditions while taking into account production requirements, and environmental conditions.

[0074] As described above, a user may be able to input environmental conditions and / or select a surface type as input conditions. In one example, a user interface may include controls to enable a user to input a set of predefined conditions (e.g., ink / surface / weather). This may allow a surface marking robot may to work in a more flexible and adaptable way, by expanding the applications including more print modes, materials, and environmental. These controls may further provide a user-friendly experience, by reducing the complexity of initiating surface printing robot workflow.

[0075] FIG. 15 illustrates a block diagram of an example computer program product 1500. In some examples, as shown in FIG. 15, computer program product 1500 includes a machine-readable storage 1502 that may also include computer readable instructions 1504. In some implementations, the machine- readable storage 1502 may be implemented as a non-transitory machine- readable storage. In an example, the computer readable instructions 1504, may be executed by a processor 1506, implement aspects of process 500 (FIG. 5),process 900 (FIG. 9), and / or process 1400 (FIG. 14). That is, printing pipeline logic illustrated in FIG. 15 may include aspects of process 500 (FIG. 5), process 900 (FIG. 9), and / or process 1400 (FIG. 14).

[0076] FIG. 16 is a block diagram illustrating a hardware apparatus including a semiconductor package according to an example. FIG. 16 shows an illustrative example of a printer 1600. In the illustrated example, the printer 1600 may include a processor 1602 and a memory 1604 communicatively coupled to the processor 1602. The memory 1604 may include computer readable instructions 1606. In an example, the computer readable instructions 1606, may be executed by the processor 1602, implement aspects of process 500 (FIG. 5) and / or process 900 (FIG. 9), described above. That is, printing pipeline logic illustrated in FIG. 16 may include aspects of process 500 (FIG. 5), process 900 (FIG. 9) and / or process 1400 (FIG. 14).

[0077] In some implementations, the processor 1602 may include a general purpose controller, a special purpose controller, a storage controller, a storage manager, a memory controller, a micro-controller, a general purpose processor, a special purpose processor, a central processor unit (CPU), the like, and / or combinations thereof. Further, implementations may include distributed processing, component / object distributed processing, parallel processing, the like, and / or combinations thereof. For example, virtual computer system processing may implement the methods or functionalities as described herein, and the processor 1602 described herein may be used to support such virtual processing.

[0078] In some examples, memory 1604 is an example of a computer- readable storage medium. For example, memory 1604 may be any memory which is accessible to the processor 1602, including, but not limited to RAM memory, registers, and register files, the like, and / or combinations thereof. References to “computer memory” or “memory” should be interpreted as possibly being multiple memories. The memory may for instance be multiple memories within the same computer system. The memory may also be multiple memories distributed amongst multiple computer systems or computing devices.

[0079] FIG. 17 shows an illustrative semiconductor apparatus 1700 (e.g., chip and / or package). The illustrated apparatus 1700 includes substrates 1702 (e.g., silicon, sapphire, or gallium arsenide) and computer readable instructions 1704 (such as, configurable computer readable instructions) and / or fixed-functionality computer readable instructions (e.g., hardware)) coupled to the substrate(s) 1702. In an example, the computer readable instructions 1704 implement aspects of process 500 (FIG. 5), process 900 (FIG. 9), and / or process 1400 (FIG. 14), described above. That is, printing pipeline logic illustrated in FIG. 16 may include aspects of process 500 (FIG. 5), process 900 (FIG. 9), and / or process 1400 (FIG. 14).

[0080] In some implementations, computer readable instructions 1704 may include transistor array and / or other integrated circuit / IC components. For example, configurable logic and / or fixed-functionality hardware logic implementations of the computer readable instructions 1704 may include configurable computer readable instructions such as, for example, programmable logic arrays (PLAs), field programmable gate arrays (FPGAs), complex programmable logic devices (CPLDs), or fixed-functionality computer readable instructions (e.g., hardware) using circuit technology such as, for example, application specific integrated circuit (ASIC), complementary metal oxide semiconductor (CMOS) or transistor-transistor logic (TTL) technology, the like, and / or combinations thereof.

[0081] All definitions, as defined and used herein, should be understood to control over dictionary definitions, definitions in documents incorporated by reference, and / or ordinary meanings of the defined terms.

[0082] Furthermore, for ease of understanding, certain functional blocks may have been delineated as separate blocks; however, these separately delineated blocks should not necessarily be construed as being in the order as discussed or otherwise presented herein. For example, some blocks may be able to be performed in an alternative ordering, simultaneously, etc.

[0083] Although a number of illustrative examples are described herein, it should be understood that numerous other modifications and examples can be devised that will fall within the spirit and scope of the principles of the foregoingdisclosure. More particularly, reasonable variations and modifications are possible in the component parts and / or arrangements of the subject combination arrangement within the scope of the foregoing disclosure, the drawings and the appended claims without departing from the spirit of the foregoing disclosure. In addition to variations and modifications in the component parts and / or arrangements, alternative uses will also be apparent. The examples may be combined to form additional examples.

Claims

CLAIMSWhat is claimed is:1 . A method of operating a surface marking robot, comprising: printing a line to a surface; capturing an image of the printed line; determining a property of the printed line based on the captured image; and in response to the property of the printed line exceeding a threshold, modifying a firing setting.

2. The method according to claim 1 , wherein modifying a firing setting includes stopping a print job.

3. The method according to claim 1 , wherein modifying a firing setting includes modifying at least one of aperture time, ink pressure, and drop spacing.

4. The method according to claim 1 , wherein modifying a firing setting includes modifying at least one of energy, voltage, and pulse width.

5. The method according to claim 1 , wherein the property includes at least one of a width of the printed line or a line quality of the printed line.

6. The method according to claim 1 , wherein determining a property includes calculating a width of the printed line based on the captured image and determining a number of pixels at a sampling point about an axis.

7. The method according to claim 1 , wherein capturing an image of the printed line includes capturing overlapping images of the printed line.

8. The method according to claim 1 , wherein modifying a firing setting includes decreasing a moving speed of the surface marking robot.

9. A surface marking robot comprising: a controller; a printing system comprising a plurality of print nozzles, wherein the print system is to deposit printing material onto a surface according to firing setting, as the surface marking robot follows an intended path; and an image capture system, wherein the image capture system is to capture an image of the printed line, wherein the controller: calculates a parameter of a printed line formed from the deposited print material based on the captured image; and in response to the parameter of the printed line exceeding a threshold, adjusts a print setting.

10. The surface marking robot of claim 9, wherein adjusting a print setting includes stopping a print job.

11. The surface marking robot of claim 9, wherein adjusting a print setting includes modifying at least one of aperture time, ink pressure, and drop spacing.

12. The surface marking robot of claim 9, wherein adjusting a print setting includes modifying at least one of energy, voltage, and pulse width.

13. The surface marking robot of claim 9, wherein the controller decreases a moving speed of the surface marking robot.

14. The surface marking robot of claim 9, wherein the parameter includes at least one of a width of the printed line or a line quality parameter of the printed line.

15. A non-transitory machine-readable storage medium comprising instructions stored thereon, wherein the instructions are executable by a processor of acontroller of a print apparatus of a surface marking robot and include instructions to: print a line to a surface; capture an image of the printed line; calculate a width of the printed line based on the captured image; and in response to the width of the printed line exceeding a threshold, modify a firing setting.

Images