Computer program, printing system, control device for printing execution unit, and printing method

Abstract

The present invention positions images with respect to a plurality of printing materials and prints the images. This control device: acquires a captured image obtained by capturing an image of an arrangement area in which M printing materials (M being an integer of 2 or more), each including a common feature part, are disposed; detects the M printing materials in the captured image; determines, on the basis of a user instruction, M printing images which are to be printed on the M printing materials, respectively, and which include two or more kinds of images from among N kinds of candidate images (N being an integer of 2 or more) associated with the feature part; positions the M printing images with respect to the detected M printing materials; and prints the M printing images onto the M printing materials.

Description

Computer program, printing system, control device for printing execution unit, and printing method

[0001] This specification relates to a technique for printing images on a plurality of printing objects.

[0002] Patent Document 1 discloses a technique for printing a target pattern on a fabric. In this technique, templates of the original pattern of the fabric and the target pattern to be printed are acquired. Then, the actual patterns on the fabric arranged on the printer are collected, and based on the deviation between the feature points of the collected actual patterns and the feature points of the original pattern, the printing position and shape of the target pattern to be printed are adjusted. The printer is controlled according to the adjusted template of the target pattern, and printing is performed on the fabric.

[0003] International Publication No. 2019 / 105481

[0004] Thus, there is a need for a technique for positioning and printing an image on a printing object such as a fabric.

[0005] This specification discloses a new technique for positioning and printing an image on a plurality of printing objects.

[0006] The technique disclosed in this specification is made to solve at least some of the above problems and can be realized as the following application examples.

[0007] [Application Example 1] A computer program that enables a computer to implement the following functions: an acquisition function that acquires a captured image obtained by photographing a placement area in which M (M is an integer of 2 or more) objects to be printed are arranged, each containing a common feature portion; a detection function that detects the M objects to be printed in the captured image; an image determination function that determines M printable images to be printed on each of the M objects from among N types (N is an integer of 2 or more) of candidate images associated with the feature portions, based on user instructions, wherein the M printable images include two or more candidate images from among the N types of candidate images; and a print control function that positions the M printable images on the detected M objects to be printed and prints the M printable images on the M objects to be printed.

[0008] According to the above configuration, from among N candidate images associated with feature portions, M printable images to be printed on M substrates are determined based on user instructions, and the M printable images are positioned and printed on the M substrates. As a result, the printable images determined from among the N candidate images can be positioned and printed on M substrates, each containing a common feature portion.

[0009] Furthermore, the technologies disclosed herein can be implemented in various forms, for example, as a control device for a printing execution unit, a printing system, a printing method, a computer program for realizing the functions of these methods, apparatuses, and systems, and a recording medium (e.g., a non-temporary recording medium) on which the computer program is stored.

[0010] A block diagram showing the configuration of the printing system 1000 of the embodiment. A perspective view showing the schematic configuration of the printing system 1000. A flowchart of the multi-product printing process. A diagram showing an example of the operation screen W1. A flowchart of the template generation process. A diagram showing an example of the feature detection screen W2. A diagram showing an example of the print dataset generation screen W3. A conceptual diagram of the template TM. A diagram showing an example of the operation screen W1 after the template TM has been generated. A flowchart of the print image determination process. A diagram showing an example of the preview screen W4. A diagram showing an example of a canvas image CI on which M print images PI are placed. A diagram explaining a modified example. A diagram showing an example of the preview screen W4x of a modified example.

[0011] A. First Embodiment: A-1: ​​Configuration of Printing System 1000 Next, embodiments will be described based on embodiments. Figure 1 is a block diagram showing the configuration of the printing system 1000 of the embodiment. The printing system 1000 includes a printer 200, a terminal device 300 as an image processing device in this embodiment, and a shooting device 400. The printer 200 and the terminal device 300, and the shooting device 400 and the terminal device 300 are connected in a communicative manner.

[0012] The terminal device 300 is a computer used by the user of the printer 200, such as a personal computer or a smartphone. The terminal device 300 includes a CPU 310 as the controller of the terminal device 300, a non-volatile storage device 320 such as a hard disk drive, a volatile storage device 330 such as RAM, an operation unit 360 that receives operations for inputting user instructions, a display unit 370 such as a liquid crystal display, and a communication unit 380. The communication unit 380 includes a wired or wireless interface for communicating with external devices, such as the printer 200 or the camera 400. The operation unit 360 may include, for example, a mouse, a keyboard, or a touch panel placed on top of the display unit 370.

[0013] The volatile storage device 330 provides a buffer area 331 for temporarily storing various intermediate data generated when the CPU 310 performs processing. The non-volatile storage device 320 stores the computer program PG1. The computer program PG1 is provided by the manufacturer of the printer 200, for example, in the form of a download from a server or stored on a DVD-ROM. The CPU 310 functions as a printer driver that controls the printer 200 by executing the computer program PG1. As a printer driver, the CPU 310 performs, for example, multi-product printing processing, which will be described later.

[0014] The imaging device 400 is a digital camera that generates image data representing a subject by optically photographing the subject. The imaging device 400 generates image data and transmits it to the terminal device 300 according to the control of the terminal device 300.

[0015] The printer 200 includes, for example, a printing mechanism 100, a CPU 210 as the controller of the printer 200, a non-volatile storage device 220 such as a hard disk drive, a volatile storage device 230 such as RAM, an operation unit 260 such as buttons or a touch panel for acquiring user input, a display unit 270 such as a liquid crystal display, and a communication unit 280. The communication unit 280 includes a wired or wireless interface for communicating with an external device, such as a terminal device 300.

[0016] The volatile storage device 230 provides a buffer area 231 for temporarily storing various intermediate data generated when the CPU 210 performs processing. The non-volatile storage device 220 stores the computer program PG2. In this embodiment, the computer program PG2 is a control program for controlling the printer 200, and may be provided stored in the non-volatile storage device 220 when the printer 200 is shipped. Alternatively, the computer program PG2 may be provided in a form that is downloaded from a server, or in a form that is stored on a DVD-ROM or the like. By executing the computer program PG2, the CPU 210 controls the printing mechanism 100 according to the print data transmitted from the terminal device 300 in the multi-product printing process described later, for example, to print an image on the printing medium. The printer 200 in this embodiment is a so-called garment printer, which is assumed to use fabric (clothing, shoes, patches described later, etc.) as the printing medium.

[0017] The printing mechanism 100 is an inkjet printing mechanism that prints by ejecting cyan (C), magenta (M), yellow (Y), black (K), and white (W) inks (droplets). The printing mechanism 100 includes a print head 110, a head drive unit 120, a main scanning unit 130, and a transport unit 140.

[0018] Further explanation will be given with reference to Figure 2. Figure 2 is a perspective view showing the schematic configuration of the printing system 1000. In Figure 2, the +X, -X, +Y, -Y, +Z, and -Z directions correspond to the left, right, front, rear, top, and bottom of the printer 200, respectively.

[0019] The main scanning unit 130, located inside the housing 201, uses the power of a main scanning motor (not shown) to reciprocate a carriage (not shown) on which the print head 110 is mounted along the main scanning direction (X direction in Figure 2). This enables main scanning, in which the print head 110 reciprocates along the main scanning direction (X direction) relative to the printing medium.

[0020] The transport unit 140 includes a platen 142 located in the center of the housing 201 in the X direction. The upper surface (the surface in the +Z direction) of the plate-shaped platen 142 is a placement area for placing the printing medium. In the example in Figure 2, M embroidered patches PT (M is an integer of 2 or more; in the example in Figure 2, there are 6 patches) are placed on the platen 142 as the printing medium. The platen 142 is transported in a transport direction (the -Y direction in Figure 2) that intersects the main scanning direction using the power of a sub-scanning motor (not shown). This enables sub-scanning, which transports the printing medium to the print head 110 in the transport direction.

[0021] The head drive unit 120 (Figure 1) drives the print head 110 by supplying a drive signal to the print head 110 while the main scanning unit 130 is performing the main scan of the print head 110. The print head 110 has a plurality of nozzles (not shown) and, in accordance with the drive signal, ejects ink onto the printing medium transported by the transport unit 140 to form dots.

[0022] As shown in Figure 2, the imaging device 400 is supported by a support (not shown) and positioned in the +Z direction of the printer 200. The imaging device 400 is located away from the printer 200 and is positioned opposite the upper surface of the platen 142, and is capable of imaging the printing medium placed on the upper surface of the platen 142. As a result, the imaging device 400 can generate data of an image including the printing medium.

[0023] A-2. Operation of Printing System 1000 The operation of the printing system 1000 will be described below. The printing system 1000 prints a predetermined image (for example, a picture or logo) onto a patch PT, which is used as a printing medium. The patch PT is placed, for example, by an operator at a predetermined position on the platen 142. At that time, it is difficult to accurately position the patch PT so that it is always in the same position on the platen 142. For this reason, the printing system 1000 has a function to detect the patch PT on the platen 142 where the patch PT is placed and to print on the detected patch PT.

[0024] A-2-1. Multi-product printing process The printing system 1000 performs a multi-product printing process to create multiple types of patch PTs by printing multiple different print images onto multiple patch PTs. Figure 3 is a flowchart of the multi-product printing process. The multi-product printing process is started when M patch PTs (6 in the example in Figure 2), which are the printing medium, are placed on the platen 142, and the patch PTs placed on the platen 142 are in a state where they can be photographed from above by the photographing device 400, and the user inputs a start command to the terminal device 300. The multi-product printing process is executed by the CPU 310 of the terminal device 300.

[0025] Here, the M patches PT are, for example, all created based on the same drawing or image. For this reason, although the M patches PT are not completely identical due to manufacturing tolerances, they are substantially identical, or at least similar to each other. For this reason, the characteristic parts of the M patches PT (for example, the edge parts described later) are common to each other.

[0026] In S110, the CPU 310 acquires data for a captured image FI showing M patches PT, which are the printing medium, from the imaging device 400. Specifically, the CPU 310 sends an imaging instruction to the imaging device 400. The imaging device 400 photographs the M patches PT placed on the platen 142, generates data for a captured image FI, and transmits this data to the terminal device 300. The data for the captured image FI is, for example, RGB image data that includes multiple RGB values ​​corresponding to multiple pixels, and the RGB values ​​indicate the color of each pixel. The RGB values ​​are color values ​​of the RGB color system that include the three component values ​​of RGB.

[0027] In S120, the CPU 310 displays an operation screen W1 that includes the captured image FI. That is, the CPU 310 uses the data of the captured image FI to generate data for the operation screen W1 that includes the captured image FI. The CPU 310 uses this data to display the operation screen W1 on the display unit 370.

[0028] Figure 4 shows an example of the operation screen W1. The operation screen W1 includes a captured image FI, buttons BT1-BT3, and slide bars SD1 and SD2. The captured image FI includes an image showing the top surface of the platen 142 and an image showing M (M=6 in the example of Figure 4) patches PT (PT1-PT6) placed on the top surface of the platen 142. Hereafter, the image showing the platen 142 included in the captured image FI will also be simply referred to as "platen 142," and the image showing the patches PT (PT1-PT6) included in the captured image FI or canvas image CI (described later) will also be simply referred to as "patches PT (PT1-PT6)."

[0029] Button BT1 is an input element for entering instructions to generate a template, which will be described later. Button BT2 is an input element for entering instructions to display the preview screen, which will be described later. Button BT3 is an input element for entering instructions to print.

[0030] Slide bars SD1 and SD2 are input elements for inputting instructions for shooting settings, which are settings related to shooting by the shooting device 400. The first slide bar SD1 is an input element for inputting instructions to set the gain level, and the second slide bar SD2 is an input element for inputting instructions to set the exposure level.

[0031] In the imaging device 400, when the light received by the image sensor is converted into an electrical signal to generate the captured image FI, the electrical signal is amplified by an amplifier. Gain is a parameter that specifies the level of amplification of the electrical signal. The larger the gain, the greater the amplification level, and therefore a brighter captured image FI is generated.

[0032] Exposure is a parameter that specifies the amount of light (light reception) that the image sensor receives. A higher exposure results in a greater amount of light reception by the image sensor, producing a brighter captured image (FI). Exposure can be adjusted, for example, by changing at least one of the shutter speed and aperture value during shooting.

[0033] When the user operates the slider bars SD1 and SD2 to input an instruction to change the shooting settings (gain and exposure in this embodiment), the terminal device 300 (CPU 310) sends a shooting instruction to the shooting device 400 to take a picture with the new shooting settings after the instruction. The shooting device 400 generates data for the captured image FI with the new shooting settings according to the shooting instruction and sends this data to the terminal device 300. The terminal device 300 updates the captured image FI in the operation screen W1 displayed on the display unit 370 using the new captured image FI data. As a result, the captured image FI in the operation screen W1 is updated in real time according to the operation of the slider bars SD1 and SD2. Note that the process of updating the operation screen W1 in response to the operation of the slider bars SD1 and SD2 is omitted in the flowchart of Figure 3.

[0034] In S130, the CPU 310 determines whether or not a template generation instruction has been entered. Specifically, it is determined that a template generation instruction has been entered when the user clicks button BT1, which is used to enter a template generation instruction, on the operation screen W1. For example, if no printing templates have been generated, the print instruction described later cannot be entered. For this reason, if no templates have been generated, the user clicks button BT1 to enter a template generation instruction.

[0035] If a template generation instruction is entered (S130: YES), the CPU 310 executes the template generation process in S140. If no template generation instruction is entered (S130: NO), the CPU 310 skips S140. The template generation process is the process of generating a template TM used for printing on materials such as patches PT.

[0036] Figure 5 is a flowchart of the template generation process. In S210, the CPU 310 acquires the current shooting setting information FS. Specifically, the CPU 310 acquires the shooting settings (gain and exposure in this embodiment) that are set at the time the template generation instruction is input as shooting setting information FS. The acquired shooting setting information FS is included in the template TM, as will be described later.

[0037] In S220, the CPU 310 determines the image to be used, which is a portion of the captured image FI, based on the user's instructions. The image to be used, which is used to generate the template TM, is the image used. For example, the CPU 310 displays a message (not shown) on the operation screen W1 prompting the user to specify the image to be used, which is the image to be used. The user, following this message, operates a pointing device such as a mouse to input an instruction to specify a portion of the captured image FI on the operation screen W1 as the image to be used, which is the image to be used, which is the message to be used. Figure 4 shows an example of the specified image to be used, which is the image to be used, which is specified to include one patch PT, as shown in Figure 4. When the CPU 310 receives an instruction to specify the image to be used, it determines the image to be used, which is the image to be used, according to the instruction.

[0038] In S225, the CPU 310 displays a feature detection screen W2, which includes the image used TI, on the display unit 370. The feature detection screen W2 is displayed in place of the operation screen W1. The feature detection screen W2 may be displayed separately from the operation screen W1. Figure 6 shows an example of the feature detection screen W2. The feature detection screen W2 in Figure 6(A) includes the image used TI, a slider bar SD3, and buttons BT4-BT6.

[0039] Button BT4 is an input element for inputting instructions for feature detection, which will be described later. Button BT5 is an input element for inputting instructions to exit the feature detection screen W2 and return to the state where the operation screen W1 is displayed. Button BT6 is an input element for inputting instructions to proceed to the next step in template generation (described later). Slide bar SD3 is an input element for inputting instructions to set the level of feature detection.

[0040] The user sets the feature detection level by operating the slider bar SD3, and then inputs a feature detection command by clicking the button BT4. Upon inputting the feature detection command, in S230, the CPU 310 detects the edges of the image TI as features of the image TI. That is, the CPU 310 generates an edge image EI that shows the edges of the image TI. For edge detection, a known method, such as Canny edge detection, is used. The parameters for edge detection are set so that the higher the set feature detection level, the more edges are detected. The parameters for edge detection include, for example, the contrast of the image TI when detecting edges and a threshold for edge intensity for detecting edges. For example, the higher the set feature detection level, the higher the contrast of the image TI is set, making it easier to detect edges. Also, the higher the set feature detection level, the lower the threshold for determining whether something is an edge is set, making it easier to detect edges.

[0041] As shown in Figure 6(B), when an edge image EI is generated from the image TI used, the edge image EI is displayed overlaid on the image TI in the feature detection screen W2. In the example in Figure 6(B), the detected edges Ei and Eo are displayed on the image TI. For example, edges Ei and Eo are displayed in a conspicuous color such as red, and the user can easily recognize the detected edges Ei and Eo by looking at the feature detection screen W2. The patch PT in this embodiment has a design embroidered on it using a single-color thread. Specifically, as shown in Figure 6, the outer shape of the patch PT is in the shape of Santa Claus. Also, the Santa Claus design is embroidered on the patch PT with white thread. In Figure 6(B), the edge Eo of the outer shape of the patch PT and the edge Ei of the design inside the patch PT are detected. If the feature detection level is too low, for example, a sufficient number of edges may not be detected, such as the main edges of the design not being detected. Furthermore, if the feature detection level is excessively high, an excessive number of edges may be detected, such as the fine edges of embroidery stitches. The user adjusts the feature detection level so that an appropriate number of edges are detected. The detected edge data, i.e., the edge image EI data, is used in the template TM described later as data indicating the feature portion of the image TI used. In other words, in this step, the CPU 310 generates edge image EI data indicating the edges of the image TI used as data indicating the feature portion of the image TI used.

[0042] Once the appropriate edge image EI (feature image) for the image TI to be used is generated, the user clicks button BT6 to advance the template generation process to the next step. When button BT6 is clicked, the CPU 310 displays the print dataset generation screen W3 in S235.

[0043] FIG. 7 is a diagram showing an example of a print data set generation screen W3. As shown in FIG. 7(A), similar to the feature detection screen W2, the print data set generation screen W3 includes a use image TI. Instead of the slide bar SD3 and the button BT4 of the feature detection screen W2, the print data set generation screen W3 includes an input field IA1 and an add button BT7. The input field IA1 is an input element for inputting the name of a print data set DS described later. The add button BT7 is an input element for inputting an addition instruction for the print data set DS.

[0044] In S240 - S260, the CPU 310 generates a print data set DS based on a user's instruction. The print data set DS is a set of data of an image to be printed (pattern image SI in this embodiment) and arrangement information AJ as described later.

[0045] In S240, the CPU 310 selects an image to be printed (pattern image SI in this embodiment) based on a user's instruction, and displays the selected pattern image SI on the print data set generation screen W3. Specifically, the user inputs the name of the print data set to be generated (in the example of FIG. 7, "Santa_red") in the input field IA1, and then clicks the add button BT7. When the add button BT7 is clicked, the CPU 310 displays a file selection screen (not shown). The user inputs a selection instruction to select a file of one pattern image SI via the file selection screen. The CPU 310 selects the pattern image SI according to the selection instruction, and displays the pattern image SI on the print data set generation screen W3.

[0046] As shown in FIG. 7(A), on the print data set generation screen W3, the selected pattern image SI is displayed overlapping the use image TI. On the print data set generation screen W3, the image information SF and the arrangement information AJ of the selected pattern image SI are displayed. In the example of FIG. 7(A), the image information SF includes the file name and the reduced image (thumbnail) of the selected pattern image SI. The arrangement information AJ is information indicating the arrangement of the pattern image SI with respect to the use image TI. The arrangement information AJ includes the coordinates (X, Y) of the pattern image SI in the coordinate system of the use image TI, the angle of the pattern image SI, and the width and height of the pattern image SI. In the state of FIG. 7(A), each value of the arrangement information AJ is an initial value.

[0047] In S250, the CPU 310 acquires the arrangement information AJ to be included in the print data set DS based on the user's instruction. Specifically, the user operates the pattern image SI displayed on the print data set generation screen W3 and arranges the object Ob of the pattern image SI at the position to be printed with respect to the printed matter (for example, the emblem PT) shown in the use image TI.

[0048] For example, in the example of FIG. 7, the object Ob of the pattern image SI is a Santa Claus with added color, and the printed matter is an emblem PT with Santa Claus embroidery. The pattern image SI is created by coloring an image used when performing embroidery on the emblem PT using a sewing machine. Therefore, by adjusting the position, size, and angle of the pattern image SI, the pattern image SI can be arranged so that the object Ob of the pattern image SI exactly overlaps the embroidery pattern of the emblem PT.

[0049] On the print dataset generation screen W3, the user can change the position, size, and angle of the pattern image SI by manipulating the border lines OF and points OP attached to the pattern image SI. The user can also change the position, size, and angle of the pattern image SI by changing the values ​​of the placement information AJ displayed on the print dataset generation screen W3. By performing these operations, the user changes the position, size, and angle of the pattern image SI on the print dataset generation screen W3 so that the object Ob of the pattern image SI perfectly overlaps with the embroidery pattern on the patch PT. In the print dataset generation screen W3 of Figure 7(B), the object Ob of the pattern image SI and the embroidery pattern on the patch PT are perfectly overlapping. With the object Ob of the pattern image SI positioned to be printed on the target material (e.g., the patch PT) shown in the image TI, the user clicks button BT6. When button BT6 is clicked, the CPU 310 acquires the arrangement information AJ of the image SI at that time as the arrangement information AJ to be included in the print dataset DS.

[0050] In S255, the CPU 310 saves (registers) a print dataset DS containing the pattern image SI and the placement information AJ. Specifically, the CPU 310 associates the data of the pattern image SI selected in S240 with the placement information AJ acquired in S255 and stores it in the non-volatile storage device 320 as a print dataset DS. As will be described in detail later, the image that is actually printed is a pattern image SI whose placement (e.g., position, size, angle) relative to the print target (in this embodiment, the patch PT) has been adjusted according to the placement information AJ. A pattern image SI whose placement (e.g., position, size, angle) relative to the print target (in this embodiment, the patch PT) has been adjusted according to the placement information AJ is also called a print image PI. In the print dataset generation screen W3 of Figure 7(B), the pattern image SI whose placement has been adjusted can be said to represent the print image PI. As can be seen from the above explanation, the print dataset DS is data that defines the print image PI and candidate images that are candidates for the print image PI.

[0051] In S260, the CPU 310 determines whether or not to terminate the generation of the print dataset DS. If the user wishes to terminate the generation of the print dataset DS, they click button BT6 on the print dataset generation screen W3. If they wish to generate more print dataset DS, they click the add button BT7. Therefore, the CPU 310 determines to terminate the generation of the print dataset DS if button BT6 is clicked, and to continue the generation of the print dataset DS if the add button BT7 is clicked.

[0052] If the generation of the print dataset DS is to continue (S260: NO), the CPU 310 returns to S240. In this way, in this embodiment, multiple print dataset DSs can be registered for a single template TM.

[0053] If the generation of the print dataset DS is terminated (S260: YES), the CPU 310 acquires print setting information PS in S270 based on the user's instructions. Specifically, the CPU 310 displays a predetermined print setting screen (not shown) on the display unit 370 for the user to input instructions to specify print settings. The user operates the print setting screen to input the print settings. The CPU 310 acquires print setting information PS according to the input. The print settings are known print settings for garment printers and include, for example, settings for whether or not to use white ink, resolution settings, and settings for whether or not to print twice. Printing twice means printing twice on the same area in order to represent darker colors.

[0054] In S280, the CPU 310 associates the acquired information with the print dataset DS to generate a template TM. Figure 8 is a conceptual diagram of the template TM. As shown in Figure 8, the template TM includes the edge image EI generated in S230 of Figure 5, the print dataset DS generated in S240-S260, the shooting setting information FS acquired in S210, and the print setting information PS acquired in S270.

[0055] Here, there is one edge image EI, one shooting setting information FS, and one printing setting information PS, but the printing dataset DS can contain N types of printing datasets (where N is an integer greater than or equal to 2) (for example, in the example in Figure 8, there are three printing datasets DSa-DSc). For example, in one template TM, N types of printing datasets DS, or in the example in Figure 8, three types of printing datasets DSa-DSc, are associated with one common information CD containing the edge image EI, shooting setting information FS, and printing setting information PS. In this way, by generating a template TM containing N types of printing datasets DS, N types of candidate images are defined that are candidates for M printing images PI to be printed for M patches PT that have edges, which are common feature parts.

[0056] As shown in Figure 8, the first print dataset DSa (named "Santa_Red") includes the first image SIa and the first placement information AJa. The second print dataset DSb (named "Santa_Blue") includes the second image SIb and the second placement information AJb. The third print dataset DSc (named "Santa_Green") includes the third image SIc and the third placement information AJc. The print images PI defined by print datasets DSa, DSb, and DSc are designated as the first print image PIa, the second print image PIb, and the third print image PIc, respectively. Print images PIa-PIc represent Santa Claus figures that have the same shape but different colors.

[0057] Once a template TM is generated, the template generation process is terminated. After the template generation process is terminated, the display unit 370 returns to the state where the operation screen W1 is displayed. Figure 9 shows an example of the operation screen W1 after a template TM has been generated. Once at least one template TM is generated, the operation screen W1 displays a display SV for selecting one or more generated templates TM, and an overview diagram HF of the selected template TM.

[0058] The display SV includes the representative image RI of the selected template TM, and selection buttons SBl and SBr for selecting one template TM from one or more generated template TMs. The representative image RI is one of the print images PI defined by one or more print datasets DS included in the selected template TM. In the example in Figure 9, the selected template TM is the template TM shown in Figure 8. In the example in Figure 9, the first print image PIa defined by the first print dataset DSaa is displayed as the representative image RI.

[0059] The overview diagram HF represents a hierarchical structure that includes a reduced representative image RMI and N types of reduced print images MIa-MICC associated with the reduced representative image RMI. The reduced representative image RMI is an image (so-called thumbnail) obtained by reducing the representative image RI mentioned above. The reduced print images MIa-MICC represent print images PIa-PICC defined by the print dataset DSa-DSc. For example, thumbnails obtained by reducing the pattern images SIa-SICC included in the print dataset DSa-DSc may be used for the reduced print images MIa-MICC. In the example in Figure 9, the reduced print image MIa is used for the reduced representative image RMI. The names of the print dataset DSa-DSb (for example, "Santa_Red") are also associated with the reduced print images MIa-MICC.

[0060] The schematic diagram HF shows, using a hierarchical structure, that the template TM in Figure 8 has a structure that includes one common information CD and N types of print datasets DS (DSa-DSc) associated with the common information CD. That is, one reduced representative image RMI corresponds to one common information CD, and N types of reduced print images MIa-MICC correspond to N types of print datasets DS (DSa-DSc). Here, a hierarchical structure means a structure having a higher-level node and one or more lower-level nodes associated with one of the higher-level nodes. The schematic diagram HF can be said to be a diagram that represents a hierarchical structure using the reduced representative image RMI corresponding to the common information CD as the higher-level node, and the reduced print images MIa-MICC corresponding to the print datasets DS (DSa-DSc) as the lower-level nodes.

[0061] Once the template generation process in S140 of Figure 3 is completed, in S150 of Figure 3, the CPU 310 determines whether or not a print command has been entered. After one or more templates TM have been generated, the user can enter a print command by clicking the print button BT3. The CPU 310 determines that a print command has been entered when the print button BT3 is clicked.

[0062] If no print command is entered (S150: NO), the CPU 310 returns to S140. In this case, the user can generate a new template TM by clicking button BT1 to enter a template generation command. If a print command is entered (S150: YES), the CPU 310 proceeds to S160.

[0063] In S160, the CPU 310 executes the object to be printed detection process. The object to be printed detection process is the process of detecting the object to be printed (in this embodiment, the patch PT) in the captured image FI based on the edge image EI included in the generated template TM.

[0064] Specifically, in S162, the CPU 310 generates an image capture edge image (not shown) that indicates the edges of the captured image FI. For example, the CPU 310 instructs the imaging device 400 to perform imaging according to the imaging setting information FS of the common information CD of the template TM, and acquires the captured image FI. The CPU 310 performs edge detection processing on the captured image FI to generate an image capture edge image. The edge detection processing is the same process used, for example, in S230 of Figure 5 when detecting the edges of the image TI used. In this embodiment, a method called Canny edge detection is used as the edge detection process.

[0065] In S164, the CPU 310 performs template matching using the edge image EI as a template image and the captured edge image as the target image. In template matching, for example, the CPU 310 searches for positions and angles in a brute-force manner, changing the position and angle of the edge image EI relative to the captured edge image, and finding positions and angles where the matching rate between the edge image EI and the captured edge image is greater than or equal to a threshold. As a result, multiple portions of the captured image FI that have edges identical or similar to the edge image EI are detected as multiple substrates.

[0066] For example, in the captured image FI, M patches PTm (where m is an integer between 1 and M, in the example in Figure 9, M=6) are detected as the printed material. In template matching, as shown by the dashed arrow AR in Figure 9, a search is performed along the first direction D1, and then the search position is shifted to the second direction D2, which is orthogonal to the first direction D1, and the next search is performed along the first direction D1. By repeating this, the entire captured image FI is searched. For this reason, in the example in Figure 9, six patches PTm are detected in the order indicated by the number "m" at the end of the patch PTm's code. Note that the search direction shown by arrow AR is just one example, and in modified versions, the search direction may be from right to left in Figure 9. Alternatively, the search may be performed in the direction along the second direction D2 (from top to bottom or from bottom to top in Figure 9) while shifting the search position along the first direction D1.

[0067] Furthermore, if two or more template TMs are generated, the detection of the object to be printed is performed using the respective edge images EI of the two or more template TMs. As will be explained in detail later, in some cases, the object to be printed corresponding to each of the two or more template TMs may be detected in the captured image FI.

[0068] In S170, the CPU 310 executes a print image determination process. The print image determination process determines M print images PI to be printed on M objects to be printed (in this embodiment, patches PTm) from among candidate images (print images) PIa-PIc defined by the print dataset DSa-DSc included in the template TM used to detect the objects to be printed.

[0069] Figure 10 is a flowchart of the print image determination process. In S310, the CPU 310 determines whether or not two or more print dataset DS are registered in the template TM used to detect the M print items. When the template TM in Figure 8 is used, there are three print dataset DS, so it is determined that two or more print dataset DS are registered.

[0070] If only one print dataset DS is registered and no more than one print dataset DS is registered (S310: NO), the CPU 310 determines in S315 that the print image PI to be printed for each printable item is the print image PI defined by that single print dataset DS. In other words, it is determined that the same print image PI will be printed on all M printable items. In this case, the process proceeds to S370 without displaying the preview screen W4 described later.

[0071] If two or more print datasets DS are registered (S310: YES), the CPU 310 determines in S320 whether the previous print count has been saved. As will be described later, the print count of each print image PI determined in the print image determination process is saved in S370 of Figure 10. In this step, it is determined whether the print count of each print image PI determined in the previous print image processing using the template TM to be used this time has been saved in the non-volatile storage device 320.

[0072] If the previous print count is saved (S320: YES), the CPU 310 retrieves the previous print count from the non-volatile storage device 320 in S325. If the previous print count is not saved (S320: NO), the CPU 310 skips S325.

[0073] In S330, the CPU 310 generates a preview image PRI, which will be described later, and displays a preview screen W4 including the preview image PRI on the display unit 370. Figure 11 shows an example of the preview screen W4. The preview screen W4 includes the preview image PRI, an overview diagram HF of the template TM used for printing, the number of detections DN, the number of print input fields NAa-NAc, a cancel button BTc, and a print button BTp.

[0074] The overview diagram HF is the same as the overview diagram HF shown on the operation screen W1 in Figure 9 described above. Specifically, the overview diagram HF includes the reduced representative image RMI, the names of the reduced print images MIa-MIC and the print dataset DS (e.g., "Santa_Red"), and represents the template TM used for printing in a hierarchical structure.

[0075] The detection count DN represents the number M (6 in the example of Figure 11) in which the object to be printed (the patch PT in the example of Figure 11) was detected in the captured image FI. The print count input fields NAa-NAc are input elements for inputting the print count of each print image PIa-PIc defined by the print dataset DSa-DSc. The initial value of the print count input fields NAa-NAc is set to the previous print count if the previous print count has been obtained in S320 of Figure 10. If the previous print count has not been obtained in S320 of Figure 10, the initial value of the print count input fields NAa-NAc is set to a predetermined value. The initial values ​​of the print count input fields NAa-NAc are adjusted according to a predetermined rule so that the sum of the initial values ​​of the print count input fields NAa-NAc matches the detection count DN. For example, the sum of the initial values ​​in the print count input fields NAa-NAc may be adjusted to match the number of detected values ​​DN by prioritizing the increase or decrease of the larger values ​​among those entered in the print count input fields NAa-NAc.

[0076] The preview image PRI includes portions of the captured image FI that contain at least M patches PT.

[0077] In S335, the CPU 310 obtains the print counts for each of the print images PIa-PIc from the print count input fields NAa-NAc. In S340, the CPU 310 determines the print image PI to be printed for each item to be printed according to the print counts of the obtained print images PIa-PIc. Specifically, the CPU 310 sequentially determines the print image PI for the M patches PTm in the order of detection (i.e., in ascending order of the last digit m). Specifically, it assigns one print image PI to be printed at a time in the detection order. For example, as shown in Figure 11, the print counts for print images PIa-PIc are 3, 2, and 1, respectively. In this case, the print image PI for the three patches PT1-PT3, which are detected in the 1st to 3rd order, is determined to be the first print image PIa. For the two patches PT4 and PT5, which are detected in the 4th and 5th positions, the printed image PI is determined to be the second printed image PIb. For the one patch PT6, which is detected in the 6th position, the printed image PI is determined to be the third printed image PIc.

[0078] In S340, the CPU 310 updates the preview image PRI displayed on the preview screen W4. Specifically, the CPU 310 positions the print image PI determined in S335 for the M patches PTm in the captured image FI included in the preview image PRI, and synthesizes the print image PI for the M patches PTm in the captured image FI.

[0079] Specifically, the object detection process in S160 of Figure 3 identifies the position of the edge image EI representing each edge of the M patches PTm in the captured image FI, that is, the positional relationship (position, size, angle) of the M edge images EI with respect to the captured image FI. The positional relationship (position, size, angle) of the pattern image SI of the print dataset DS that defines the printed image PI with respect to the edge image EI is indicated by the arrangement information AJ of the print dataset DS that defines the printed image PI. For this reason, for example, the CPU 310 can position the first printed image PIa (i.e., the pattern image SIa with adjusted position, size, and angle) relative to the patches PT1 in the captured image FI based on the detection result of the patches PT1 in the captured image FI and the first arrangement information AJa included in the first print dataset DSaa. Similarly, the second print image PIb is positioned relative to the patch PT4 in the captured image FI based on the detection result of the patch PT4 in the captured image FI and the second placement information AJb included in the second print dataset DSb.

[0080] The CPU 310 generates a preview image PRI by compositing a printed image PI at a position determined for each of the M patches PTm in the captured image FI, and displays the preview image PRI on the preview screen W4. For example, in the example shown in Figure 11, the first printed image PIa is composited onto three patches PT1-PT3 in the preview image PRI. Then, the second printed image PIb is composited onto two patches PT4 and PT5, and the third printed image PIc is composited onto one patch PT6.

[0081] In S350, the CPU 310 determines whether the print count entered in the print count input fields NAa-NAc on the preview screen W4 has been updated. If the user operates the operation unit 360 and changes at least one of the print counts entered in the print count input fields NAa-NAc, it is determined that the print count entered in the print count input fields NAa-NAc has been updated. If the print count has been updated (S350: YES), the CPU 310 returns to S335. As a result, the print image PI to be printed for M patches PT is determined again according to the print count, and the preview image PRI is updated (S335-S345).

[0082] If the print count is not updated (S350: NO), the CPU 310 determines in S360 whether or not a print command has been entered. The user can enter a print command by clicking the print button BTp on the preview screen W4. The CPU 310 determines that a print command has been entered when the print button BTp is clicked. If no print command is entered (S360: NO), the CPU 310 returns to S350. If a print command is entered (S360: YES), the CPU 310 proceeds to S370.

[0083] In S370, the CPU 310 saves the determined print count and terminates the print image determination process. For example, if, without displaying the preview screen W4, it is determined in S315 to print a print image PI defined by one print dataset DS for M patches PT, the print count for that one print image PI is saved. Also, if the preview screen W4 is displayed and the print count for each of the print images PI (e.g., PIa-PIc) defined by two or more print dataset DS included in the template TM to be used is determined, the print count for each of the two or more print images PI is saved. The saved print counts can be obtained in S325 of the next print image determination process.

[0084] Although a flowchart is omitted, when the preview button BT2 on the operation screen W1 is clicked, the print image determination process in Figure 10 is executed with steps S310 and S315 omitted. In other words, in this case, even if only one print dataset DS is registered in the template TM to be used, the preview screen W4 is always displayed.

[0085] Once the print image determination process is complete, in step S180 of Figure 3, the CPU 310 generates print data. Specifically, the CPU 310 prepares a canvas image CI in memory (volatile storage device 330) that corresponds to the area on the platen 142 where the M patches PT are placed. The CPU 310 positions M print images PI to be printed for the M patches PT on the canvas image CI and places the M print images PI at the positioned locations.

[0086] Figure 12 shows an example of a canvas image CI on which M print images PI are arranged. The canvas image CI contains M print images PI determined for each of the M patches PT in the print image determination process. As shown in Figure 12, the first print image PIa is placed at the positions on the canvas image CI corresponding to patches PT1-PT3 on the platen 142 shown in the captured image FI (Figure 11, etc.) described above. Similarly, the second print image PIb is placed at the positions on the canvas image CI corresponding to patches PT4 and PT5, and the third print image PIc is placed at the position on the canvas image CI corresponding to patch PT6. The position, size, and angle of each print image PI are determined by the position of the edge of the patch PT relative to the captured image FI detected by the print object detection process (position of the edge image EI), and the arrangement information AJ of the print dataset DS that defines the print image PI, similar to the position, size, and angle of the print image PI synthesized into the preview image PRI described above.

[0087] The CPU 310 generates print data by performing known color conversion and halftone processing on a canvas image CI on which M print images PI are arranged. The color conversion process converts the value of each pixel of RGB image data from the color values ​​of the RGB color system to the color values ​​of a color system whose components are the ink colors (e.g., CMYK values) using a lookup table. The halftone processing converts the color-converted image data into print data (also called dot data) that shows the formation state of print dots for each pixel, using known methods such as dithering or error diffusion.

[0088] In S190, the CPU 310 supplies the generated print data to the printer 200. The printer 200 performs printing according to the received print data. As a result, the print image PI is printed on each of the M printable objects, in this embodiment, the M patches PT, placed on the platen 142, and M colored patches PT are created. For example, if the canvas image CI of Figure 12 is used, three types of patches are created: three patches with the first print image PIa printed on them, two patches with the second print image PIb printed on them, and one patch with the third print image PIc printed on it.

[0089] According to the embodiment described above, the CPU 310 acquires a captured image FI obtained by photographing a placement area (for example, the upper surface of the platen 142) on which M printable objects (in this embodiment, patch PTs) are arranged, each containing a common feature portion (for example, an edge) (S110 in Figure 3). The CPU 310 detects the M patch PTs in the captured image FI (S160 in Figure 3). The CPU 310 determines, based on user instructions, M printable images PI to be printed on the M patch PTs from among N types of candidate images (for example, printable images PIa-PIc) associated with the edge image EI representing the feature portion, which is an edge (S170 in Figure 3). In the examples of Figures 11 and 12, the M printable images PI to be determined include two or more images from among the N types of candidate images, printable images PIa-PIc. The CPU 310 positions M print images PI for each of the M detected patch PTs and prints the M print images PI for each of the M patch PTs (S180, S190 in Figure 3, and Figure 12). As a result, it is possible to position and print a print image determined from N types of candidate images for each of the M patch PTs, each containing a common feature portion (for example, identical or similar feature portions). This makes it easy to create multiple types of patch PTs, for example, multiple patch PTs with different colors or designs.

[0090] Furthermore, according to this embodiment, the CPU 310 displays a preview screen W4 (Figure 11) on the display unit 370, which includes reduced print images MIa-MIc showing N types of candidate images associated with a single edge image EI, and print count input fields NAa-NAc for inputting user instructions (S330 in Figure 10). Based on the user instructions input to the preview screen W4, the CPU 310 determines M print images PI to be printed on M patches PT (S335, S340 in Figure 10). The M print images PI to be printed can be easily determined based on the user instructions input to the preview screen W4 which includes N types of candidate images. Therefore, for example, a user can easily create multiple types of patches PT.

[0091] More specifically, the preview screen W4 includes print count input fields NAa-NAc for inputting the number of prints for each of the N types of candidate images (i.e., print images PIa-PIc) (Figure 11). The CPU 310 determines M print images PI from the N types of candidate images to be printed on M patches PT according to the print counts entered in the print count input fields NAa-NAc (S340 in Figure 10). As a result, the user can easily create multiple types of patches PT in the desired number simply by inputting the number of prints.

[0092] Furthermore, according to this embodiment, the CPU 310 adds a print dataset DS to be associated with the common information CD in the template generation process in response to user instructions (S240-S260 in Figure 5). The print dataset DS is data that defines candidate images (e.g., print images PIa-PIc), and the common information CD includes edge images EI that show feature portions. In other words, S240-S260 in Figure 5 can be described as the process of adding candidate images (e.g., print images PIa-PIc) to be associated with the edge images EI that show feature portions. The CPU 310 displays a preview screen W4 containing N types of candidate images, including the added print images, on the display unit 370 (S330 in Figures 11 and 10). As a result, candidate images to be associated with edge images EI that show feature portions can be added in response to user instructions, which is convenient for the user.

[0093] Furthermore, according to this embodiment, the preview screen W4 includes a screen (Schematic diagram HF in Figure 11) showing a hierarchical structure that includes a reduced representative image RMI corresponding to common information CD such as the edge image EI, and N types of reduced print images MIa-MICC associated with the reduced representative image RMI. As a result, N types of candidate images (print images PIa-PIC) associated with common feature parts are clearly displayed on the preview screen W4, so for example, the user can appropriately input the number of prints for each candidate image while looking at the preview screen W4. Therefore, the user can more easily create multiple types of patches PT in the desired number.

[0094] Furthermore, according to this embodiment, the preview screen W4 includes a display of the number of patches PT detected in the captured image FI (i.e., the number of detected patches DN in Figure 11). As a result, the user can input the number of prints into the print count input fields NAa-NAc while keeping track of the number of patches PT created by printing, which is convenient for the user.

[0095] Furthermore, according to this embodiment, the CPU 310 saves the number of prints for each determined print image PIa-PIc in the print image determination process (S370 in Figure 10). In the print image determination process for the next multi-product printing process, the CPU 310 retrieves the previously saved print count (S325 in Figure 10) and displays this print count as the initial value of the print count input field NAa-NAc on the preview screen W4 (S330 in Figure 10). Therefore, if the user clicks the print button BTp without changing the print count from the initial value, they can print M patches PT with the same print count as the previous time. In other words, the CPU 310 can determine the M print images to be printed in the (k+1)th and subsequent multi-product printing processes according to the print count entered for the kth (k is an integer of 1 or more) multi-product printing process. As a result, for example, if the process of printing M print images PI for M patches PT is repeated with the same number of prints for each candidate image, the user does not need to enter the number of prints in the print count input fields NAa-NAc each time a multi-product printing process is performed. Therefore, the user's workload can be reduced.

[0096] Furthermore, in this embodiment, the CPU 310 detects M patches PTm in the print detection process (S160 in Figure 3) by shifting the search position in the second direction D2 while searching in the first direction D1, as shown by the arrow AR in Figure 9. Then, one print image PI is assigned to each patch in the detection order (i.e., in ascending order of the digit m at the end of the code in Figure 9). For this reason, if the M print image PIs include two or more images of the same type, the M print image PIs are determined such that two or more patch PTs with two or more images of the same type printed on them are not scattered but are arranged together. For example, if the M print image PIs include two or more images of the same type, the M print image PIs are likely to be determined such that two or more adjacent patch PTs have those two or more images of the same type printed on them. For example, in the canvas image CI of Figure 12, the three first print images PIa are adjacent to each other, and the two second print images PIb are adjacent to each other. As a result, for example, handling the created patch PT becomes easier than when patch PTs of different types are arranged randomly. For example, it becomes easier to check the number of each type of created patch PT, and to store and ship the created patch PTs together by type.

[0097] Furthermore, in this embodiment, when the CPU 310 receives a print command on the operation screen W1 (YES in S150 of Figure 3), if two or more print data sets DS are registered, that is, if two or more candidate images are registered (YES in S310 of Figure 10), the CPU 310 displays the preview screen W4 (S330 in Figure 10), and determines M print images PI to be printed on M patches PT based on the user's instructions input to the preview screen W4 (specifically, the number of prints for each print image PIa-PIc) (S335-S340 in Figure 10). As a result, if user instructions are required, the CPU 310 can obtain user instructions by displaying the preview screen W4 and determine the M print images PI.

[0098] Furthermore, in this embodiment, when the CPU 310 receives a print command on the operation screen W1 (YES in S150 of Figure 3), if only one print dataset DS is registered, that is, if only one type of candidate image is registered and two or more types of candidate images are not registered (NO in S310 of Figure 10), the CPU 310 does not display the preview screen W4 and, regardless of the user's instructions, determines that each of the M print images to be printed on the M patches PT will be one type of candidate image defined by the one print dataset DS (S315 of Figure 10). If there is only one type of candidate image, there is no choice, so it is considered acceptable to determine the M print images PI to be that one candidate image. In this embodiment, when user instructions are not required, the display of the preview screen W4 is omitted and the M print images PI are automatically determined, thus reducing the user's effort.

[0099] Furthermore, according to this embodiment, the common feature portion of the M patches PT is the edges Ei and Eo (Figure 6). The CPU 310 acquires an edge image EI showing the edges Ei and Eo of one patch PT (S230 in Figure 5), generates an captured edge image showing the edges in the captured image FI (S162 in Figure 3), and detects the M patches PT by searching for the edge image EI of the feature portion of the patch PT in the captured edge image (S164 in Figure 3). As a result, the M patches PT in the captured image FI can be appropriately detected using the edge image EI showing the edges Ei and Eo of one patch PT.

[0100] As can be seen from the above explanation, the edges shown in the edge image EI are examples of feature parts, and the preview screen W4 is an example of the input screen.

[0101] B. Modifications (1) In the above embodiment, the print image PI printed on one patch PT is one image (specifically, a pattern image SI with adjusted placement), so one print dataset DS contains one image (pattern image SI). Alternatively, the print image printed on one substrate may contain two or more images. In this case, as described below, one print dataset is generated to contain two or more images.

[0102] Figure 13 is an explanatory diagram of a modified example. Figure 13(A) shows a modified patch PTv. The modified patch PTv includes a first part Ast having the shape and embroidery of a Santa Claus design, and a second part Atx for printing letters.

[0103] Figure 13(B) shows an example of the print dataset generation screen W3 that is displayed when generating a print dataset DSv that defines a print image containing two or more images to be printed for one patch PTv in Figure 13(A). As shown in the image information SF and used image TI of the print dataset generation screen W3, in this modified example, a picture image SIst showing a picture of Santa Claus and a character image SItx showing text are selected as images to be printed. That is, in S240 of Figure 5 of the template generation process, the CPU 310 selects the picture image SIst and the character image SItx based on the user's instructions. Then, in S250 of Figure 5, the CPU 310 obtains the placement information AJst and AJtx of the picture image SIst and character image SItx, respectively, based on the user's instructions. For example, the placement information AJst of the picture image SIst is obtained so that the object showing Santa Claus in the picture image SIst is placed in the first part Ast of the patch PTv. Then, placement information AJtx for the character image SItx is obtained so that the objects representing the characters in the character image SItx are placed in the second part Atx of the patch PTv. The print dataset generation screen W3 in Figure 13(B) shows the print image PIv, which includes the adjusted placement of the character image SItx and the pattern image SIst.

[0104] Figure 13(C) shows an example of a print dataset DSv generated in a modified example. The print dataset DSv in Figure 13(C) is a dataset that defines the print image PIv shown on the print dataset generation screen W3 in Figure 13(B). At S255 in Figure 5, the CPU 310 generates a print dataset DSv (Figure 13(C)) which includes data for a pattern image SIst, placement information AJst for the pattern print image SIst, data for a character image SItx, and placement information AJtx for the character print image SItx. As a result, a print image PIv containing two images SIst and SItx is printed for one patch PTv.

[0105] As can be seen from the above explanation, as in this modified example, the printable image to be printed on a single substrate (e.g., a patch PTv), and the candidate images that are candidates for that printable image, may be a combination of two or more images (e.g., a combination of a picture image PIST and a text image PITx). As a result, a wider variety of images can be printed on substrates such as patches. Therefore, a variety of products, such as colored patches, can be created.

[0106] (2) In the above embodiment, the case in which multiple types of colored patches are created by printing one of the different print images PIa-PIC on one type of uncolored patch PT was described. However, the invention is not limited to this, and printing may be performed on two or more types of uncolored patches in a single multi-product printing process.

[0107] Figure 14 shows an example of a preview screen W4x of a modified example. In the example in Figure 14, printing is performed on four patches PT (PT1-PT4) similar to those in the first embodiment, and two patches PTx (PTx1, PTx2) which have a different shape and embroidery from the patches PT. For this purpose, the platen 142 is arranged with four patches PT (PT1-PT4) and two patches PTx (PTx1, PTx2).

[0108] In this case, during the template generation process at S140 in Figure 3, the CPU 310 generates both a first template for the embroidered PT and a second template for the embroidered PTx as templates TM. Then, during the print detection process at S160 in Figure 3, the CPU 310 detects the embroidered PT (PT1-PT4 in Figure 14) in the captured image FI by performing template matching using the edge image EI included in the first template. The CPU 310 also detects the embroidered PTx (PTx1, PTx2 in Figure 14) in the captured image FI by performing template matching using the edge image EI included in the second template.

[0109] In the print image determination process at S170 in Figure 3, the CPU 310 displays the preview screen W4x in Figure 14 on the display unit 370 (S330 in Figure 10). In this modified example, since there are two templates TM to be used, the preview screen W4x displays the schematic diagrams HF and HFx of the two templates TM. The first template is the same as the template TM in Figure 8 of the embodiment. For this reason, the schematic diagram HF of the first template is the same as the schematic diagram HF of the preview screen W4 in Figure 11.

[0110] The overview diagram HFx of the second template represents a hierarchical structure that includes a reduced representative image RMIx and two types of reduced print images Mid and MIe associated with the reduced representative image RMI. The reduced print images MIe and Mid are thumbnails representing the print images PIe and PId defined by the print dataset (not shown) of the second template. The reduced representative image RMIx is a thumbnail representing the representative image (print image PId) of the second template. The reduced print images MIe and Mid are also associated with the name of the print dataset (for example, "cat_red").

[0111] The preview screen W4x displays the number of detected patch PTs DN using the first template and the number of detected patch PTx DNx using the second template. The preview screen W4x also includes print count input fields NAa-NAc for print images PIa-PIc defined by the print dataset included in the first template, and print count input fields NAd and NAe for print images PIe and PId defined by the print dataset included in the second template. By entering numerical values ​​in the print count input fields NAa-NAe, the user can specify the number of prints for each of the print images PIa-PIc to be printed on patch PT1-PT3, and the number of prints for each of the candidate images PId and PIe to be printed on patch PTx1 and PTx2. In this way, printing can be performed on two or more types of uncolored patches in a single multi-product printing process.

[0112] (3) In the above embodiment, the N candidate images, print images PIa-PIc, are images of Santa Claus with different colors. That is, the print images PIa-PIc are images that are different from each other. However, the N candidate images may be images that are identical in image form, with only their placement relative to the edge image EI (in other words, their placement relative to the printed material such as a patch PT) being different from each other. For example, the N candidate images may include a specific image to be printed at a first position on a predetermined printed material and a specific image to be printed at a second position different from the first position on the predetermined printed material. For example, each of the N candidate images is defined by a single print dataset DS, as described in the embodiment. The N candidate images only need to differ in at least one of the images included in the print dataset DS that defines each candidate image (e.g., pattern image SI) and the placement information AJ. Thus, generally speaking, the N candidate images only need to differ in at least one of the placement of the candidate image relative to feature parts such as edges and the candidate image itself.

[0113] (4) In the above embodiment, the schematic diagram HF of the operation screen W1 (Figure 9) and the preview screen W4 (Figure 11) uses a reduced representative image RMI as an image that represents the higher-level node for expressing the hierarchical structure, that is, an image corresponding to the common information CD of the template TM. However, the image that represents the higher-level node may be a text image such as the name of the template TM, or a simple graphic image such as a circle or a dot.

[0114] Similarly, in the overview diagram HF, an image representing a lower node to represent the hierarchical structure, i.e., an image corresponding to the print dataset DS of the template TM, is used to represent the print image PI defined by the print dataset DS. Specifically, a reduced print image (e.g., MIa-MIC) obtained by reducing the pattern image SI included in the print dataset DS is used. However, the image representing the lower node may be, for example, a text image such as the name of the print dataset DS, or an image of a simple shape such as a circle or a dot.

[0115] Instead of the template TM overview diagram HF, a non-hierarchical structure, such as a list of available print datasets DS, may be displayed on the operation screen W1 or the preview screen W4.

[0116] (5) In the above embodiment, the CPU 310 determines the M print images PI to be printed for the M patches PT according to the print counts entered in the print count input fields NAa-NAc for each candidate image on the preview screen W4 (S335-S340 in Figure 10). Alternatively, for example, the CPU 310 may determine the M print images PI to be printed for the M patches PT by having the user select one image to be printed from among multiple types of candidate images PIa-PIc for each of the M detected patches PT.

[0117] (6) In the above embodiment, an embroidered patch PT is used as the substrate. However, it is not limited to this, and various other materials can be used as the substrate. For example, other textile products such as handkerchiefs may be used as the substrate. Alternatively, acrylic products such as acrylic sheets used for keychains, badges, acrylic stands, etc. may be used as the substrate.

[0118] Furthermore, the printed material is not limited to embroidered patches; other embroidered products such as handkerchiefs, hats, and shoes may also be used.

[0119] Furthermore, the printed material is not limited to the above-mentioned textile products, acrylic products, and embroidered products; other articles and parts, such as paper products, resin products other than acrylic, and metal products, may also be used.

[0120] (7) In the above embodiment, the edges Ei and Eo of the patch PT, which are detected by performing edge detection processing on the image TI used, are used as the feature portion of the patch PT (Figure 6, etc.). The feature portion of the printed material, such as the patch PT, may be a portion different from the edge. For example, the feature portion of the patch PT may be a specific portion of the image TI used that is specified by the user (all or part of the image TI used). In this case, for example, the template TM includes the data of the specific portion as the feature portion data, instead of the data of the edge image EI. Then, in the printed material detection processing, for example, the specific portion is used as a template, and the captured image FI is used as the target image, and template matching is performed to detect M patch PTs in the captured image FI.

[0121] Alternatively, the feature portion of the patch PT may be multiple feature points detected by applying a predetermined detection algorithm to the image TI used. The predetermined detection algorithm may be, for example, the MSER algorithm or corner detection algorithms such as FAST or Harris. In this case, for example, the template TM includes the coordinates and feature quantities of the detected feature points as feature portion data. For example, feature quantities called SIFT (Scale-Invariant Feature Transform) or SURF (Speeded-Up Robust Features) may be used for the feature quantities of the feature points. In the print detection process, for example, the CPU 310 may detect M patches PT by performing feature point matching in the captured image FI to search for feature points that are identical or similar to multiple feature points of the image TI used.

[0122] (8) The specific processing details of the multi-product printing process and the specific contents of screens W1-W4 in the above embodiment are examples and may be omitted or changed as appropriate. For example, the display of the number of detections DN in the preview screen W4 may be omitted. For example, in the template generation process of Figure 5, the CPU 310 may automatically perform edge detection processing on the image TI used to generate the edge image EI without displaying the feature detection screen W2 (Figure 6). For example, in the print image determination process of Figure 10, steps S310 and S315 in Figure 10 may be omitted. That is, the CPU 310 may always display the preview screen W4 even if only one print dataset DS is registered in the template TM to be used. Also, steps S320 and S325 in Figure 10 may be omitted. In other words, the CPU 310 does not save the number of prints for each print image PIa-PIc from the previous print run, and may always set the initial values ​​of the print count input fields NAa-NAc on the preview screen W4 to predetermined fixed values ​​(for example, 0 or 1).

[0123] (9) In the print image determination process shown in Figure 10 of the above embodiment, if two or more print data sets DS are registered in the template TM to be used, the CPU 310 displays a preview screen W4 and has the user specify the number of prints for each of the print images PIa-PIc defined by the two or more print data sets DS. Alternatively, for example, the number of patches PT to be printed in one multi-product printing process M and the number of prints for each of the print images PIa-PIc may be registered in advance in the template TM. In this case, for example, if the CPU 310 detects a number of patches PT of the number registered in advance M in the captured image FI, it may determine the M print images PI to be printed on the M patches PT according to the number of prints registered in advance in the template TM, without displaying the preview screen W4.

[0124] (10) In the above embodiment, the placement area on which the material to be printed is placed and photographed by the photographing device 400 is the upper surface of the platen 142, but is not limited to this. For example, the placement area may be the upper surface of the conveyor belt. Also, for example, if the material to be printed is a part of a cloth (for example, embroidery), the placement area may be a part of the cloth.

[0125] (11) The device that performs all or part of the multi-product printing process shown in Figure 3 may be any other device instead of the terminal device 300. For example, the CPU 210 of the printer 200 or the CPU of the imaging device 400 may perform the multi-product printing process shown in Figure 3. In this case, the terminal device 300 is not necessary.

[0126] Furthermore, the device that performs the multi-product printing process in Figure 3 may be a server connected via the Internet to at least one of the printer 200, terminal device 300, and imaging device 400. In this case, the server may be a so-called cloud server composed of multiple computers that can communicate with each other. For example, when a server connected to the printer 200 performs the multi-product printing process in Figure 3, for example, when the server displays the operation screen W1 in S120 of Figure 3, it generates the data for the operation screen W1 and sends this data to the printer 200, thereby displaying the operation screen W1 on the display unit 270 of the printer 200. Also, instructions entered by the user for the operation screen W1 are acquired by the printer 200 and sent to the server. In other words, the server acquires user instructions via the printer 200. The same applies when displaying the feature detection screen W2 and the print data set generation screen W3 in S225 and S235 of Figure 5, and when displaying the preview screen W4 in S330 of Figure 10.

[0127] Furthermore, the multi-product printing process shown in Figure 3 may be divided and executed by multiple devices. For example, the multi-product printing process shown in Figure 3 may be divided and executed by the printer 200 and the server connected to the printer 200, or by the terminal device 300 and the server connected to the terminal device 300. For example, in the multi-product printing process shown in Figure 3, the template generation process in S140 may be mainly executed by the printer 200 or the terminal device 300, while the print target detection process, print image determination process, and print data generation processes in S160-S180 may be mainly executed by the server.

[0128] (12) In each of the above embodiments, a part of the configuration that was implemented by hardware may be replaced with software, or conversely, a part or all of the configuration that was implemented by software may be replaced with hardware.

[0129] Furthermore, if some or all of the functions of the present invention are implemented by a computer program, the program may be provided in the form of a computer-readable recording medium (for example, a non-temporary recording medium). The program may be used while stored on the same or a different recording medium (computer-readable recording medium) as it was provided. "Computer-readable recording medium" is not limited to portable recording media such as memory cards and CD-ROMs, but may also include internal storage devices within a computer, such as various ROMs, and external storage devices connected to a computer, such as hard disk drives.

[0130] The present invention has been described above based on examples and modifications. However, the embodiments of the invention described above are for the purpose of facilitating understanding of the present invention and do not limit it. The present invention can be modified and improved without departing from its spirit, and equivalents thereof are included.

[0131] 1000...Printing system, 100...Printing mechanism, 110...Print head, 120...Head drive unit, 130...Main scanning unit, 140...Transport unit, 142...Platen, 200...Printer, 201...Housing, 210...CPU, 220...Non-volatile memory device, 230...Volatile memory device, 260...Operation unit, 270...Display unit, 280...Communication unit, 300...Terminal device, 310...CPU, 320...Non-volatile memory device, 330...Volatile memory device, 360...Operation unit, 370...Display unit, 380...Communication unit, 400...Photography Shadow device, AJ... Placement information, DN... Number of detections, DS... Print data set, EI... Edge image, FI... Captured image, FS... Capture setting information, HF... Overview diagram, PG1, PG2... Computer program, PI... Print image, PRI... Preview image, PS... Print setting information, PT... Patch, RI... Representative image, RMI... Reduced representative image, SF... Image information, TI... Image used, TM... Template, W1... Operation screen, W2... Feature detection screen, W3... Print data set generation screen, W4... Preview screen

Claims

1. A computer program that enables a computer to implement the following functions: an acquisition function that acquires a captured image obtained by photographing a placement area in which M (M is an integer of 2 or more) objects to be printed are arranged, each containing a common feature portion; a detection function that detects the M objects to be printed in the captured image; an image determination function that determines M printable images to be printed on each of the M objects based on user instructions, from among N (N is an integer of 2 or more) candidate images associated with the feature portions, wherein the M printable images include two or more candidate images from among the N candidate images; and a print control function that positions the M printable images relative to the detected M objects to be printed and prints the M printable images on the M objects to be printed.

2. A computer program according to claim 1, further comprising N candidate images associated with one of the feature portions, a computer implementing a screen display function that displays an input screen for inputting user instructions on a display unit, and the image determination function determining the M printable images to be printed on the M printable materials based on the user instructions input to the input screen.

3. A computer program according to claim 2, wherein the computer implements an additional function to add candidate images corresponding to the feature portion in response to user instructions, and the screen display function displays the input screen including the N types of candidate images, including the added candidate images.

4. A computer program according to claim 2, wherein the input screen includes a screen showing a hierarchical structure including a higher-level node corresponding to a common feature portion and the N types of candidate images associated with the higher-level node.

5. A computer program according to claim 4, wherein in the input screen, the higher-level node is represented by a representative image selected from the N types of candidate images.

6. A computer program according to claim 2, wherein the input screen further includes an input field for inputting the number of prints for each of the N types of candidate images, and the image determination function determines the M print images to be printed on the M printable materials from among the N types of candidate images according to the number of prints entered in the input field.

7. A computer program according to claim 6, wherein the input screen further includes displaying the number of printed materials detected in the captured image.

8. A computer program according to claim 6, further comprising: a computer function that enables the computer to store the print count for each of the N types of candidate images; and an image determination function that determines the M print images to be printed in the (k+1)th and subsequent print processes according to the print counts that have been input and stored for the kth print process (where k is an integer of 1 or more).

9. A computer program according to claim 2, wherein one candidate image includes a combination of two or more images.

10. A computer program according to claim 6, wherein the image determination function determines the M printable images to be printed on the M printed materials such that, if the M printable images include two or more images of the same type, two or more images of the same type are printed on two or more adjacent printed materials among the M printed materials.

11. A computer program according to claim 2, wherein if two or more candidate images are registered at the time the print instruction is received, the screen display function displays the input screen, and the image determination function determines the M print images to be printed on the M printable materials based on the user's instructions entered on the input screen.

12. A computer program according to claim 11, wherein, when a print instruction is received, one type of candidate image is registered and two or more types of candidate images are not registered, the screen display function does not display the input screen, and the image determination function determines that each of the M print images to be printed on the M printable materials is one type of candidate image, regardless of the user's instructions.

13. A computer program according to claim 1, wherein the feature portion includes an edge, and the detection function detects the M printed objects by acquiring the edge of one of the printed objects, generating an edge image showing the edge in the captured image, and searching for the edge of the feature portion in the edge image.

14. A computer program according to claim 1, wherein the N types of candidate images differ from each other in at least one of the arrangement of the candidate images relative to the feature portion and the candidate images themselves.

15. A computer program according to any one of claims 1 to 14, wherein the printed material is a textile product or an acrylic product.

16. A computer program according to claims 1 to 14, wherein the printed material is an embroidered product.

17. A printing system comprising a computer and a printing device, wherein the computer includes: an acquisition unit that acquires a captured image obtained by photographing a placement area on which M (M is an integer of 2 or more) objects to be printed are arranged, each containing a common feature portion; a detection unit that detects the M objects to be printed in the captured image; an image determination unit that, based on user instructions, determines M printable images to be printed on each of the M objects from among N types (N is an integer of 2 or more) of candidate images associated with the feature portions, wherein the M printable images include two or more candidate images from among the N types of candidate images; and a generation unit that positions the M printable images on the detected M objects to be printed and generates print data for printing the M printable images on the M objects to be printed; and the printing device prints the M printable images on the M objects to be printed using the print data.

18. A control device for a print execution unit, comprising: an acquisition unit that acquires a captured image obtained by photographing an arrangement area on which M (M is an integer of 2 or more) objects to be printed are arranged, each containing a common feature portion; a detection unit that detects the M objects to be printed in the captured image; an image determination unit that, based on user instructions, determines M print images to be printed on each of the M objects to be printed, from among N (N is an integer of 2 or more) candidate images associated with the feature portions, wherein the M print images include two or more of the N candidate images; and a print control unit that positions the M print images on the detected M objects to be printed and prints the M print images on the M objects to be printed.

19. A printing method comprising: an acquisition step of acquiring a captured image obtained by photographing a placement area on which M objects to be printed are arranged, each containing a common feature portion; a detection step of detecting the M objects to be printed in the captured image; an image determination step of determining, based on user instructions, M printable images to be printed on each of the M objects to be printed, from among N types of candidate images (N being an integer of 2 or more) associated with the feature portions, wherein the M printable images include at least two of the N types of candidate images; and a print control step of positioning the M printable images relative to the detected M objects to be printed, and printing the M printable images on the M objects to be printed.

Images