Computer program, image processing device, and image processing method

Abstract

To accurately determine relative positional relationship between images.SOLUTION: A computer program causes a computer to execute: processing to specify a plurality of first edge pixels in an object image by using object image data, and specify a plurality of second edge pixels in a template image by using template image data; processing to determine relative positional relationship between the object image and the template image with first accuracy; processing to determine first use pixels out of the first edge pixels and determine second use pixels out of the second edge pixels on the basis of positional relationship between the first edge pixels and the second edge pixels in the determined positional relationship; and processing to determine relative positional relationship between the object image and the template image with second accuracy higher than the first accuracy by using the first use pixels and the second use pixels.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present specification relates to techniques for determining relative positional relationships between images. [Background technology]

[0002] The image measuring device described in Patent Document 1 obtains the approximate position and orientation of the mark in the captured image by pattern matching between a captured image obtained by photographing a workpiece with a mark provided thereon using a low-resolution camera and a template image. This image measuring device then obtains the position and orientation of the mark in the captured image with even greater accuracy based on the measured edge points on the mark in the captured image and the known positional relationship of the mark shape. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2006-113931 A [Patent Document 2] JP 2010-197336 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the above technique may not be able to identify an object in an image with sufficient accuracy. For example, the above technique is based on the premise that the exact shape of the mark is known. For this reason, for example, when there are individual differences in the shapes of objects in the images, there is a possibility that the relative positional relationship between the images may not be determined with sufficient accuracy.

[0005] This specification discloses a new technique that can determine the relative positional relationship between images with high accuracy. [Means for solving the problem]

[0006] The technology disclosed in this specification can be realized in the following application examples.

[0007] [Application Example 1] A computer program comprising: Contains objects an image acquisition process for acquiring target image data indicating a target image and template image data indicating a template image corresponding to the object; an edge image specification process for specifying a plurality of first edge pixels in the target image using the target image data and specifying a plurality of second edge pixels in the template image using the template image data; a first determination process for determining a relative positional relationship between the target image and the template image with a first accuracy; and a pixel-to-be-used determination process for determining a plurality of first use pixels from a plurality of first edge pixels and determining a plurality of second use pixels from the plurality of second edge pixels, wherein a sum of the plurality of first use pixels and the plurality of second use pixels is less than a sum of the plurality of first edge pixels and the plurality of second edge pixels; and a second determination process for determining a relative positional relationship between the target image and the template image with a second accuracy higher than the first accuracy, using the plurality of first use pixels and the plurality of second use pixels.

[0008] According to the above configuration, after the relative positional relationship between the target image and the template image is determined with a first accuracy, a plurality of first use pixels are determined from a plurality of first edge pixels, and a plurality of second use pixels are determined from a plurality of second edge pixels based on the positional relationship. Then, using the plurality of first use pixels and the plurality of second use pixels, the relative positional relationship between the target image and the template image is determined with a second accuracy higher than the first accuracy. As a result, when determining the relative positional relationship between the target image and the template image with the second accuracy, it is possible to suppress the inconvenience that the positional relationship cannot be determined with sufficient accuracy because unnecessary edge pixels are used. Therefore, it is possible to accurately specify the relative positional relationship between the target image and the template image. Furthermore, it is possible to suppress the inconvenience that the processing time of the second determination process increases because unnecessary edge pixels are used. Therefore, it is possible to reduce the time required to specify an object in an image. [Application example 2] The computer program according to Application Example 1, A computer program, wherein the first determination process is executed using reduced target image data generated by performing a reduction process on the target image data, and reduced template image data generated by performing a reduction process on the template image data. [Application example 3] The computer program according to application example 1 or 2, a computer program product, wherein in the use pixel determination process, the plurality of first use pixels are determined from among pixels included in a partial image that is a part of the target image that corresponds to the template image and is determined based on the positional relationship determined in the first determination process. [Application example 4] The computer program according to Application Examples 1 to 3, a computer program, wherein the pixel to be used determination process includes at least one of a first process for determining, among the plurality of first edge pixels, pixels having a corresponding second edge pixel as the first pixel to be used, and not determining, as the first pixel to be used, pixels having no corresponding second edge pixel, and a second process for determining, among the plurality of second edge pixels, pixels having a corresponding first edge pixel as the second pixel to be used, and not determining, as the second pixel to be used, pixels having no corresponding first edge pixel. [Application example 5] The computer program according to Application Example 4, a second edge pixel corresponding to a particular first edge pixel is the second edge pixel that exists within a predetermined distance from the particular first edge pixel in the positional relationship determined in the first determination process. [Application Example 6] The computer program according to application example 4 or 5, a computer program product, in which, in the used pixel determination process, if the number of pixels among the plurality of first edge pixels included in a partial image which is a portion of the target image corresponding to the template image and which is determined based on the positional relationship determined in the first determination process is greater than the number of the second edge pixels, the first process is executed, and if the number of pixels among the plurality of first edge pixels included in a partial image which is a portion of the target image corresponding to the template image and which is determined based on the positional relationship determined in the first determination process is less than the number of the second edge pixels, the second process is executed. [Application Example 7] The computer program according to any one of Application Examples 1 to 6, The first determination process is a pattern matching process. [Application Example 8] The computer program according to any one of Application Examples 1 to 7, The second determination process is a process using an iterative closest point (ICP) algorithm.

[0009] The technology disclosed in this specification can be realized in various forms, for example, in the form of an image processing method, an image processing device, a computer program for realizing the functions of the method or device, a recording medium on which the computer program is recorded (e.g., a non-temporary recording medium), etc. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram showing the configuration of a printing system 1000 according to an embodiment. [Diagram 2] FIG. 1 is a perspective view showing a schematic configuration of a printing system 1000. [Diagram 3] 4 is a flowchart of a template registration process and a print process. [Figure 4] FIG. 4 is a diagram showing an example of an image used in the present embodiment. [Diagram 5] 13 is a flowchart of a matching process. [Figure 6] FIG. 4 is a diagram showing an example of an edge image used in the matching process. [Figure 7] 13 is a flowchart of a valid edge pixel determination process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] A. First Example: A-1: Configuration of the printing system 1000 Next, an embodiment will be described based on an example. Fig. 1 is a block diagram showing the configuration of a printing system 1000 of the example. The printing system 1000 includes a printer 200, a terminal device 300 as an image processing device of the example, and an imaging device 400. The printer 200 and the terminal device 300, and the imaging device 400 and the terminal device 300 are connected so as to be able to communicate with each other.

[0012] The terminal device 300 is a computer used by a user of the printer 200, such as a personal computer or a smartphone. The terminal device 300 includes a CPU 310 as a controller of the terminal device 300, a non-volatile storage device 320 such as a hard disk drive, a volatile storage device 330 such as a RAM, an operation unit 360 such as a mouse or a keyboard, a display unit 370 such as a liquid crystal display, and a communication unit 380. The communication unit 380 includes a wired or wireless interface for communicatively connecting to an external device, such as the printer 200 or the imaging device 400.

[0013] The volatile storage device 330 provides a buffer area 331 for temporarily storing various intermediate data generated when the CPU 310 performs processing. The non-volatile storage device 320 stores a computer program PG1. The computer program PG1 is provided by the manufacturer of the printer 200, for example, in a form downloaded from a server or stored on a DVD-ROM or the like. The CPU 310 executes the computer program PG1 to function as a printer driver that controls the printer 200. The CPU 310 as the printer driver executes, for example, a template registration process and a printing process, which will be described later.

[0014] The imaging device 400 is a digital camera that optically captures an image of a subject to generate image data (also called captured image data) representing the subject. The imaging device 400 generates the captured image data and transmits it to the terminal device 300 under the control of the terminal device 300.

[0015] The printer 200 includes, for example, a print mechanism 100, a CPU 210 as a controller for the printer 200, a non-volatile storage device 220 such as a hard disk drive, a volatile storage device 230 such as a RAM, an operation unit 260 such as buttons or a touch panel for acquiring operations by a user, a display unit 270 such as a liquid crystal display, and a communication unit 280. The communication unit 280 includes a wired or wireless interface for communicatively connecting to an external device, for example, a terminal device 300.

[0016] The volatile storage device 230 provides a buffer area 231 for temporarily storing various intermediate data generated when the CPU 210 performs processing. The non-volatile storage device 220 stores a computer program PG2. In this embodiment, the computer program PG2 is a control program for controlling the printer 200, and may be stored in the non-volatile storage device 220 when the printer 200 is shipped and provided. Alternatively, the computer program PG2 may be provided in a form downloaded from a server, or in a form stored in a DVD-ROM or the like. By executing the computer program PG2, for example, the CPU 210 controls the printing mechanism 100 according to print data transmitted from the terminal device 300 by a printing process described later, and prints an image on a print medium. Note that the printer 200 of this embodiment assumes fabric as the print medium, and can print an image on, for example, clothing S such as a T-shirt (see FIG. 2).

[0017] The printing mechanism 100 is an inkjet printing mechanism that performs printing by ejecting ink (droplets) of each of the colors C, M, Y, and K. The printing mechanism 100 includes a print head 110, a head driving unit 120, a main scanning unit 130, and a transport unit 140.

[0018] Further description will be made with reference to Fig. 2. Fig. 2 is a perspective view showing a schematic configuration of the printing system 1000. The +X direction, -X direction, +Y direction, -Y direction, +Z direction, and -Z direction in Fig. 2 are the left, right, front, rear, upper, and lower sides of the printer 200, respectively.

[0019] The main scanning unit 130 uses the power of a main scanning motor (not shown) inside the housing 201 to reciprocate a carriage (not shown) carrying the print head 110 along the main scanning direction (X direction in FIG. 2). This achieves main scanning by reciprocating the print head 110 along the main scanning direction (X direction) relative to a print medium such as clothing S.

[0020] The transport unit 140 includes a platen 142 and a tray 144 provided at the center of the housing 201 in the X direction. The top surface (surface in the +Z direction) of the plate-shaped platen 142 is a placement surface for placing a print medium such as clothes S. The platen 142 is fixed to a plate-shaped tray 144 arranged in the -Z direction of the platen 142. The tray 144 is one size larger than the platen 142. The platen 142 and the tray 144 hold the print medium such as clothes S. The platen 142 and the tray 144 are transported in a transport direction (Y direction in FIG. 2) intersecting the main operation direction by using the power of a sub-scanning motor (not shown). This realizes a sub-scan that transports the print medium such as clothes S in the transport direction relative to the print head 110.

[0021] The head drive unit 120 (FIG. 1) supplies a drive signal to the print head 110 to drive the print head 110 while the main scanning unit 130 is performing a main scan of the print head 110. The print head 110 has a plurality of nozzles (not shown), and ejects ink to form dots on a printing medium transported by a transport unit 140 in accordance with the drive signal.

[0022] 2, the imaging device 400 is supported by a support (not shown) and disposed in the +Z direction of the printer 200. The imaging device 400 is located at a location away from the printer 200, is provided facing the upper surface of the platen 142, and is capable of capturing an image of a print medium such as clothing S placed on the upper surface of the platen 142. This allows the imaging device 400 to generate captured image data that shows an image including a print medium such as clothing S.

[0023] A-2. Operation of the Printing System 1000 The operation of the printing system 1000 will be described. The printing system 1000 prints a predetermined image (for example, a pattern or a logo) on a print target area, which is a part of the clothing S as a print medium. In this embodiment, as shown in FIG. 2, the clothing S is a T-shirt, and the print target area is an area including a chest pocket PC provided on the T-shirt. The clothing S is placed on the platen 142, for example, by an operator. At that time, it is difficult to accurately place the clothing S on the platen so that the chest pocket PC is always placed in the same position on the platen 142. For this reason, the printing system 1000 has a function of specifying, as the print target area on the platen 142 on which the clothing S is placed, an area including the chest pocket PC, and performing printing on the specified print target area.

[0024] A-2-1. Template registration process The template registration process is a process for generating template image data for identifying a print target area in which a chest pocket PC is located in a print process described below, using a sample garment S. The sample garment S is, for example, one of multiple garments S to be printed.

[0025] 3(A) is a flowchart of the template registration process. The template registration process is started when, for example, sample clothing S is placed on platen 142 and the clothing S placed on platen 142 is in a state where it can be photographed from above by imaging device 400, and the user inputs a start command to terminal device 300. The template registration process is executed by CPU 310 of terminal device 300.

[0026] In S100, the CPU 310 acquires captured image data of the sample clothing S from the imaging device 400. Specifically, the CPU 310 transmits an imaging instruction to the imaging device 400. The imaging device 400 captures an image of the sample clothing S placed on the platen 142, generates captured image data, and transmits the captured image data to the terminal device 300. The captured image data is image data that includes, for example, a plurality of RGB values ​​corresponding to a plurality of pixels, and indicates the color of each pixel by the RGB values. The RGB values ​​are color values ​​of the RGB color system that include three component values ​​of RGB. The captured image data of the sample clothing S acquired in this step is also called sample image data, and the image indicated by the sample image data is also called a sample image.

[0027] Fig. 4 is a diagram showing an example of an image used in this embodiment. As shown in Fig. 4, the left-right direction of the image corresponds to the main scanning direction of the printer 200 (X direction in Fig. 2), and the up-down direction of the image corresponds to the transport direction of the printer 200 (Y direction in Fig. 2). Fig. 4(A) shows an example of a sample image It shown by sample image data. The sample image It includes an image showing a sample garment S placed on the platen 142. Hereinafter, the image showing the sample garment S included in the sample image It will be simply referred to as "garment St", and the image showing the chest pocket PC will be simply referred to as "chest pocket PCt".

[0028] In S105, the CPU 310 crops the print target area PAt in the sample image It based on a user instruction. For example, the CPU 310 displays a user interface (UI) screen (not shown) on the display unit 370. The user inputs an instruction to specify the print target area PAt in the sample image It on the UI screen using a pointing device such as a mouse. In the example of FIG. 4(A), the print target area PAt including the breast pocket PCt of the clothing St is illustrated. The CPU 310 executes a cropping process on the sample image data to generate partial sample image data corresponding to the print target area PAt.

[0029] In S110, the CPU 310 stores the partial sample image data as template image data in the non-volatile storage device 220. Fig. 4(B) shows an example of a template image TI represented by the template image data (partial sample image data).

[0030] A-2-2. Printing process The printing process is a process in which a predetermined image (e.g., a pattern or logo) is printed on a print target area, which is a part of the clothing S as a printing medium, using the printer 200. FIG. 3(B) is a flowchart of the printing process. The printing process is started when the clothing S as a printing medium is placed on the platen 142, and the user inputs a start command to the terminal device 300 in a state in which the clothing S placed on the platen 142 can be photographed from above by the imaging device 400. The printing process is executed by the CPU 310 of the terminal device 300.

[0031] In S200, the CPU 310 acquires captured image data of the clothing S, which is a print medium, from the imaging device 400. The acquisition method is the same as that in S100 of Fig. 3(A). The captured image data of the clothing S, which is a print medium, acquired in this step is also called medium image data, and the image shown by the medium image data is also called a medium image.

[0032] 4(D) shows an example of a medium image Is shown by the medium image data. The medium image Is includes an image showing a sample of clothing S placed on the platen 142, similar to the sample image It. Hereinafter, the image showing the clothing S included in the medium image Is will be simply referred to as "clothing Ss," and the image showing the chest pocket PC will be simply referred to as "chest pocket PCs."

[0033] In S210, the CPU 310 executes a matching process to identify the print target area PAs in the medium image Is. The matching process is executed using the medium image data and the template image data. The matching process is a process for determining the positional relationship between the medium image Is and the template image TI. The details of the matching process will be described later.

[0034] The positional relationship between the two images Is and TI is indicated, for example, by the position (coordinates) of the template image TI relative to the medium image Is and the inclination (angle) of the template image TI relative to the medium image Is. The positional relationship may further include the size (scale) of the template image TI relative to the medium image Is. In FIG. 4(D), the rectangle indicated by the dashed line indicates the template image TI arranged on the medium image Is in the positional relationship determined by the matching process. The CPU 310 specifies the print target area PAs based on the specified positional relationship. For example, the area of ​​the template image TI superimposed on the medium image Is in the specified positional relationship (FIG. 4(D)) is specified as the print target area PAs.

[0035] In S215, the CPU 310 positions the print image (e.g., a pattern or logo) based on the specified print target area PAs and executes printing. For example, the CPU 310 generates print data and transmits it to the printer 200 so that the print image is printed in an area on the garment S that corresponds to the print target area PAs specified in the medium image Is (e.g., an area of ​​a chest pocket PC). The printer 200 controls the printing mechanism 100 according to the received print data to print the image on the garment S.

[0036] A-2-3. Matching process The matching process in S210 in FIG. 3B will be described. FIG. 5 is a flowchart of the matching process. In S300, CPU 310 executes a reduction process on the template image data to generate reduced template image data. The reduction process is a process for reducing the number of pixels in the X direction and the Y direction of the image to be processed at a predetermined reduction ratio. The reduction process is executed using a known method, for example, a nearest neighbor method, a bilinear method, or a bicubic method. The predetermined reduction ratio is, for example, 10 to 50%.

[0037] In S305, an edge detection process is performed on the reduced template image data to generate reduced template edge data. The edge detection process is a process for detecting edge pixels from among a plurality of pixels in the image to be processed, and generating edge data indicating the detection result. The edge data is binary image data indicating whether each pixel is an edge pixel or a non-edge pixel.

[0038] The edge pixel detection method may be various methods. In this embodiment, the Canny Edge method is used, which is preferable for detecting edge pixels that indicate the contour of an object in an image. Alternatively, for example, a method may be used in which edge strength is calculated using a Laplacian filter or a Sobel filter, and pixels whose edge strength is greater than a threshold value TH1 are detected as edge pixels.

[0039] 4C shows a reduced template edge image MEIt represented by the reduced template edge data. The reduced template edge image MEIt includes the edge MEt of the contour of the breast pocket PCt in the template image TI of FIG. 4B.

[0040] In S310, the CPU 310 executes a reduction process on the medium image data to generate reduced medium image data. The reduction process in this step uses the same process as the reduction process in S300. The reduction ratio in the reduction process in this step is the same as the reduction ratio in the reduction process in S300.

[0041] In S315, edge detection processing is performed on the reduced medium image data to generate reduced medium edge data. The edge detection processing uses the same detection method as the edge detection processing in S305.

[0042] FIG. 4E shows a reduced media edge image MEIs represented by the reduced media edge data. The reduced media edge image MEIs includes edges MEs of the contours of the clothes Ss and the chest pocket PCs in the media image Is of FIG. 4D.

[0043] In S320, the CPU 310 executes matching with a rough accuracy by pattern matching. Specifically, the CPU 310 uses the reduced medium edge data and the reduced template edge data to determine a rough positional relationship between the reduced medium edge image MEIs and the reduced template edge image MEIt. A known algorithm is used for the pattern matching algorithm. For example, the pattern matching is a method of calculating the similarity between the two images MEIs and MEIt in the area where the reduced medium edge image MEIs and the reduced template edge image MEIt overlap while changing the positional relationship (coordinates and angle) between the reduced medium edge image MEIs and the reduced template edge image MEIt by a predetermined increment, and searching for the positional relationship with the highest similarity. For example, the similarity between the two images MEIs and MEIt is the number of pixels that overlap the edge pixels of the reduced template edge image MEIt among the multiple edge pixels of the reduced medium edge image MEIs.

[0044] The accuracy of matching is determined by the increment. In this step, the increment of coordinates is set to, for example, one to several pixels, and the increment of angle is set to, for example, 3 to 7 degrees.

[0045] In FIG. 4(E), the rectangle indicated by the dashed line indicates the reduced template edge image MEIt placed on the reduced medium edge image MEIs in a positional relationship determined by pattern matching.

[0046] In S325, the CPU 310 performs edge detection processing on the original template image data that has not been reduced, to generate template edge data. The edge detection processing uses the Canny Edge method, as in the edge detection processing in S300. Alternatively, a method using various detection filters or the like may be used.

[0047] Fig. 6 is a diagram showing an example of an edge image used in the matching process. Fig. 6(A) shows a template edge image TEI indicated by template edge data. In the template edge image TEI, edge pixels Egt included in the template image TI in Fig. 4(B) are specified.

[0048] In S330, the CPU 310 crops the portion of the medium image Is before reduction to be processed based on the result of the pattern matching performed using the reduced image in S320. That is, the CPU 310 specifies the positional relationship between the medium image Is before reduction and the template image TI based on the positional relationship between the reduced medium edge image MEIs and the reduced template edge image MEIt specified in S320. The CPU 310 crops the area of ​​the medium image Is that overlaps with the template image TI in the specified positional relationship to generate partial media image data.

[0049] 4(F) shows a partial medium image SI represented by partial medium image data generated by cropping. The partial medium image SI shows a portion of the medium image Is that includes the chest pocket PCs.

[0050] In S335, the CPU 310 performs edge detection processing on the partial media image data generated by cropping to generate partial media edge data. The edge detection processing uses the Canny Edge method, as in the edge detection processing in S300. Alternatively, a method using various detection filters or the like may be used.

[0051] 6B shows a partial media edge image SEI indicated by the partial media edge data. In the partial media edge image SEI, edge pixels Egs included in the partial media image SI in FIG. 4F are identified.

[0052] In S340, the CPU 310 executes a valid edge pixel determination process. The valid edge pixel determination process is a process for determining edge pixels (also called valid edge pixels) to be used in the final matching performed in S345 from among a plurality of edge pixels Egt of the template image TI specified in the template edge image TEI and a plurality of edge pixels Egs of the partial medium image SI specified in the partial medium edge image SEI.

[0053] 7 is a flowchart of the effective edge pixel determination process. In S410, the CPU 310 counts the number Nt of the edge pixels Egt of the template image TI using the template edge data. In S415, the CPU 310 counts the number Ns of the edge pixels Egs of the partial medium image SI using the partial medium edge data.

[0054] In S420, the CPU 310 determines whether the number Ns of edge pixels Egs of the partial medium image SI is greater than the number Nt of edge pixels Egt of the template image TI.

[0055] If the number Ns of edge pixels Egs of the partial medium image SI is greater than the number Nt of edge pixels Egt of the template image TI (S420: YES), in S425 the CPU 310 determines the template image TI as the comparison image, and determines the partial medium image SI as the reference image.

[0056] If the number Ns of edge pixels Egs of the partial medium image SI is less than or equal to the number Nt of edge pixels Egt of the template image TI (S420: NO), in S430, the CPU 310 determines the template image TI as the reference image, and determines the partial medium image SI as the comparison image.

[0057] In S435, one edge pixel of interest is selected from among the multiple edge pixels in the reference image determined in S425 or S430. In S440, CPU 310 identifies coordinates in the comparison image that correspond to the edge pixel of interest in the reference image (hereinafter also referred to as corresponding coordinates). The positional relationship between the reference image and the comparison image has been determined by pattern matching in S320 of FIG. 5. In S445, CPU 310 uses edge data of the comparison image to search for an edge pixel within a circle of a predetermined radius R centered on the corresponding coordinates in the comparison image. Radius R is, for example, several pixels to several tens of pixels.

[0058] In S450, CPU 310 determines whether or not there is an edge pixel within the search range based on the search result in S445. If there is an edge pixel within the search range (S450: YES), in S455 CPU 310 determines the edge pixel of interest in the reference image as a valid edge pixel. If there is no edge pixel within the search range (S450: NO), in S460 CPU 310 determines the edge pixel of interest in the reference image as an invalid edge pixel. Note that the edge pixel determined as an invalid edge pixel is removed from the reference image, that is, the edge pixel is changed from an edge pixel to a non-edge pixel in the reference image. The edge pixel determined as a valid edge pixel is maintained as an edge pixel in the reference image.

[0059] In S465, CPU 310 determines whether all edge pixels in the reference image have been processed as edge pixels of interest. If there are unprocessed edge pixels in the reference image (S465: NO), CPU 310 returns to S435 and selects the unprocessed edge pixel as a new edge pixel of interest. If all edge pixels in the reference image have been processed (S465: YES), CPU 310 ends the valid edge pixel determination process.

[0060] The edge data of the reference image in which the number of edge pixels in the reference image is reduced by the valid edge pixel determination process is generated. For example, when the template image TI is the reference image, the processed template edge data is generated. When the partial medium image SI is the reference image data, the processed partial medium edge data is generated. In the example of FIG. 6, as can be seen from the template edge image TEI in FIG. 6(A) and the partial medium edge image SEI in FIG. 6(B), the template image TI is the reference image, and the partial medium image SI is the comparison image. For this reason, in the example of FIG. 6, the processed template edge data showing the processed template edge image LEI shown in FIG. 6(C) is generated. In the processed template edge image LEI, the edge pixels surrounded by the dashed lines C1, C2, and C3 among the edge pixels Egt in the template image TI (template edge image TEI) are excluded as invalid edge pixels. This is because there are no edge pixels in the vicinity of the positions of the partial medium image SI (partial medium edge image SEI) corresponding to these edge pixels.

[0061] When the effective edge pixel determination process is completed, in S345 of FIG. 5, the CPU 310 executes matching between the partial medium image SI and the template image TI using an ICP (iterative closest point) algorithm. Specifically, the matching is executed using effective edge pixels of the reference image of the template image TI and the partial medium image SI, and all edge pixels of the comparison image as pixels to be used. In the example of FIG. 6, the matching is executed using a plurality of edge pixels Egs specified in the partial medium edge image SEI of FIG. 6(B) and a plurality of edge pixels Egt of the processed template edge image LEI of FIG. 6(C) as pixels to be used. The matching is executed by performing an optimization calculation according to the ICP algorithm so that the distance between the edge pixels of the two images is minimized.

[0062] The matching in this step is performed with higher accuracy than the pattern matching in S320 in Fig. 5, with the positional relationship determined by the pattern matching in S320 in Fig. 5 as the initial position. For example, the matching in this step is performed with an accuracy of less than one pixel for coordinates (offset) and an accuracy of less than one degree for angles.

[0063] In S350, CPU 310 judges whether the average error of the matching in S345 is equal to or less than a threshold value TH. The average error is the average of the distances between corresponding edge pixels of the two images in the positional relationship after matching. If the average error is equal to or less than the threshold value TH, it can be judged that the matching in S345 was performed with a sufficiently high accuracy.

[0064] If the average matching error is equal to or less than the threshold value TH (S350: YES), in S360, the CPU 310 determines a print target area PAs (FIG. 4(D)) in the medium image Is based on the result of the pattern matching in S320 and the result of the ICP matching in S345. Specifically, the CPU 310 specifies the final positional relationship between the medium image Is and the template image TI based on the sum of the offset amounts of the two matchings and the sum of the rotation angles. The CPU 310 determines the area on the medium image Is where the template image TI overlaps in the final positional relationship as the print target area PAs.

[0065] If the average error of matching is greater than the threshold value TH (S350: NO), in S355 the CPU 310 executes an interruption process to interrupt the matching process. The interruption process may, for example, involve displaying on the display unit 370 a message indicating that the process is being interrupted and a message instructing the operator to start over from the step of setting the garment S on the platen 142.

[0066] According to the present embodiment described above, the CPU 310 identifies a plurality of edge pixels Egs (FIG. 6(B)) in the medium image Is using the medium image data (S335 in FIG. 5). The CPU 310 identifies a plurality of edge pixels Egt (FIG. 6(A)) in the template image TI using the template image data (S325 in FIG. 5). The CPU 310 determines the relative positional relationship between the medium image Is and the template image TI with a relatively rough first accuracy (S320 in FIG. 5). The CPU 310 determines a plurality of use pixels from among the edge pixels Egs, Egt based on the positional relationship with the edge pixels Egs, Egt in the determined positional relationship (S340 in FIG. 5). The use pixels in this embodiment are the pixels determined to be valid edge pixels among the edge pixels of the reference image and all edge pixels of the comparison pixels. The use pixels are less than the sum of the edge pixels Egs, Egt. That is, at least a part of the edge pixels Egs, Egt are excluded from the use pixels. The CPU 310 uses the determined use pixels to determine the relative positional relationship between the medium image Is and the template image TI with a second accuracy higher than the first accuracy (S345 in FIG. 5). As a result, when determining the relative positional relationship between the medium image Is and the template image TI with the second accuracy, it is possible to suppress the inconvenience of not being able to determine the positional relationship with sufficient accuracy due to the use of unnecessary edge pixels. Therefore, it is possible to accurately specify the relative positional relationship between the medium image Is and the template image TI. Furthermore, it is possible to suppress the inconvenience of an increase in the processing time of the matching process in FIG. 5 due to the use of unnecessary edge pixels. Therefore, for example, it is possible to reduce the time required to specify an object (for example, chest pocket PCs) in the medium image Is.

[0067] Furthermore, according to this embodiment, the determination of the positional relationship with the first accuracy in S320 is performed using reduced medium image data (S310 in FIG. 5) generated by performing a reduction process on the medium image data, and reduced template image data (S300 in FIG. 5) generated by performing a reduction process on the template image data. As a result, the processing time required to determine the positional relationship with a relatively low accuracy can be reduced.

[0068] Furthermore, according to this embodiment, as shown in Figure 6(B), the used pixels of the medium image Is (edge ​​pixels Egs in Figure 6(B)) are determined from among the pixels contained in the partial medium image SI (Figure 4(F)), which is a part of the medium image Is that corresponds to the template image TI.

[0069] According to the above configuration, the pixels to be used in the medium image Is are determined from the pixels contained in the partial medium image SI, so that the number of pixels to be used can be reduced compared to when the pixels are determined from the entire medium image Is. Therefore, the processing time required for matching in S345 can be further reduced.

[0070] Furthermore, according to this embodiment, in the valid edge pixel determination process of Fig. 7, CPU 310 determines, among the edge pixels of the reference image, pixels having a corresponding edge pixel of the comparison image as pixels to be used (valid edge pixels), and does not determine, as pixels to be used, pixels having no corresponding edge pixel (S450 to S455 in Fig. 7). As a result, the number of pixels to be used can be appropriately reduced. As a result, the processing time required for matching in S345 can be further reduced.

[0071] Here, the edge pixel of the comparison image corresponding to the specific edge pixel of the reference image is specifically an edge pixel of the comparison image that exists within a predetermined distance (in this embodiment, radius R) from the specific edge pixel of the reference image in the positional relationship determined by rough matching between the reference image and the comparison image (S440, S445 in FIG. 7). As a result, it is possible to easily determine whether or not there is an edge pixel of the comparison image that corresponds to the specific edge pixel of the reference image.

[0072] Furthermore, according to this embodiment, in the valid edge pixel determination process of Fig. 7, if the number Ns of edge pixels Egs of the partial medium image SI (partial medium edge image SEI) is greater than the number Nt of edge pixels Egt of the template image TI (template edge image TEI) (YES in 420 of Fig. 7), a process is executed in which the partial medium image SI is used as the reference image (S425 of Fig. 7). If the number Ns is less than the number Nt (NO in 420 of Fig. 7), a process is executed in which the template image TI is used as the reference image (S430 of Fig. 7). As a result, the number of pixels used in an image that contains a relatively large number of edge pixels can be appropriately reduced.

[0073] Furthermore, according to this embodiment, the coarse matching in S325 of Fig. 5 is a pattern matching process, which allows high-speed, coarse matching.

[0074] Furthermore, according to this embodiment, the high-precision matching in S345 in Fig. 5 is a process that uses an iterative closest point (ICP) algorithm, and as a result, high-precision matching can be achieved using appropriate edge pixels.

[0075] As can be seen from the above explanation, the edge pixels Egs of the partial medium image SI (partial medium edge image SEI) are an example of the first edge pixels, and the edge pixels Egt of the template image TI (template edge image TEI) are an example of the second edge pixels. The matching of S320 in Fig. 5 is an example of the first determination process, and the matching of S345 in Fig. 5 is an example of the second determination process.

[0076] B. Modifications (1) In the above embodiment, the medium image data showing the clothing S is used as the target image data to identify the area to be printed in the clothing S. This is not limited to this, and can be used for various other purposes. For example, in order to identify the area to be printed in a sheet of paper, or to identify a marker showing the area to be printed in the sheet of paper, scan data obtained by scanning the sheet of paper may be used as the target image data. Furthermore, when identifying an industrial part in a captured image, in order to operate the industrial part with a robot, the captured image data may be used as the target image data.

[0077] (2) In the above embodiment, the pattern matching in S320 of Fig. 5 is performed using reduced medium image data and reduced template image data. Alternatively, the pattern matching in S320 of Fig. 5 may be performed using non-reduced medium image data and template image data.

[0078] (3) In the above embodiment, the processes of S335 to S345 in Fig. 5 are executed using partial media image data, which is a part of the medium image data. Alternatively, for example, when the difference in size between the medium image data and the template image data is small, the processes of S335 to S345 in Fig. 5 may be executed using the medium image data.

[0079] (4) In the above embodiment, of the partial medium image SI and the template image TI, an image with a larger number of edge pixels is used as a reference image, and an image with a smaller number of edge pixels is used as a comparison image, and the processes of S435 to S465 in Fig. 7 are executed. Alternatively, for example, after the processes of S435 to S465 are executed with one of the partial medium image SI and the template image TI as a reference image and the other as a comparison image, the processes of S435 to S465 may be further executed with the other of the partial medium image SI and the template image TI as a reference image and the other as a comparison image.

[0080] (5) In S445 of the above embodiment, the search range is set to a circle with a radius R centered on the edge pixel of interest. This is not limiting, and for example, the reference image may be divided into a plurality of blocks, and the block in which the edge pixel of interest is located may be set to the search range.

[0081] (6) In the above embodiment, pattern matching is used for the matching in S320 in Fig. 5. Instead of this, matching using other algorithms may be performed. As other algorithms, for example, Chamfer Matching may be used.

[0082] (7) In the above embodiment, the ICP algorithm is used for the matching in S345 in Fig. 5. Instead of this, matching using another algorithm may be performed.

[0083] (8) The device that executes all or part of the template registration process in FIG. 3(A) and the print process in FIG. 3(B) may be various other devices instead of terminal device 300. For example, CPU 210 of printer 200 or a CPU of imaging device 400 may execute the template registration process in FIG. 3(A) and the print process in FIG. 3(B). In this case, terminal device 300 is not required. In addition, the device that executes the template registration process in FIG. 3(A) and the print process in FIG. 3(B) may be a server connected to at least one of printer 200, terminal device 300, and imaging device 400 via the Internet. In this case, the server may be a so-called cloud server configured by a plurality of computers that can communicate with each other.

[0084] (9) In each of the above embodiments, part of the configuration realized by hardware may be replaced by software, and conversely, part or all of the configuration realized by software may be replaced by hardware.

[0085] Furthermore, when some or all of the functions of the present invention are realized by a computer program, the program can be provided in a form stored in a computer-readable recording medium (e.g., a non-transitory recording medium). The program can be used in a state stored in the same or a different recording medium (computer-readable recording medium) from when it was provided. "Computer-readable recording medium" is not limited to portable recording media such as memory cards and CD-ROMs, but can also include internal storage devices within a computer, such as various ROMs, and external storage devices connected to a computer, such as a hard disk drive.

[0086] The present invention has been described above based on examples and modifications, but the above-described embodiments of the invention are intended to facilitate understanding of the present invention and are not intended to limit the present invention. The present invention may be modified or improved without departing from the spirit of the invention, and equivalents thereof are included in the present invention. [Explanation of symbols]

[0087] 100...printing mechanism, 1000...printing system, 110...print head, 120...head driving section, 130...main scanning section, 140...conveying section, 142...platen, 144...tray, 200...printer, 201...casing, 210...CPU, 220...non-volatile storage device, 230...volatile storage device, 231...buffer area, 260...operation section, 270...display section, 280...communication section, 300...terminal device, 310...CPU, 320...non-volatile storage memory device, 330...volatile memory device, 360...operation unit, 370...display unit, 380...communication unit, 400...imaging device, Is...media image, It...sample image, LEI...processed partial media edge image, MEIs...reduced media edge image, MEIt...reduced template edge image, PG1, PG2...computer program, SEI...partial media edge image, SI...partial media image, TEI...template edge image, TI...template image

Claims

1. A computer program comprising: an image acquisition process for acquiring target image data representing a target image including an object and template image data representing a template image corresponding to the object; an edge image identification process that uses the target image data to identify a plurality of first edge pixels in the target image and uses the template image data to identify a plurality of second edge pixels in the template image; a first determination process for determining a relative positional relationship between the target image and the template image with a first accuracy; a pixel-to-be-used determination process for determining a plurality of first use pixels from among the plurality of first edge pixels and determining a plurality of second use pixels from among the plurality of second edge pixels based on a positional relationship between the plurality of first edge pixels and the plurality of second edge pixels in the positional relationship determined in the first determination process, wherein a sum of the plurality of first use pixels and the plurality of second use pixels is less than a sum of the plurality of first edge pixels and the plurality of second edge pixels; a second determination process for determining a relative positional relationship between the target image and the template image with a second accuracy higher than the first accuracy, using the plurality of first use pixels and the plurality of second use pixels; Run the following on your computer: a computer program product, the pixel to be used determination process including at least one of a first process, prior to the second determination process, for determining, among the plurality of first edge pixels, pixels having a corresponding second edge pixel as the first pixel to be used, and not determining, as the first pixel to be used, pixels having no corresponding second edge pixel, and a second process, prior to the second determination process, for determining, among the plurality of second edge pixels, pixels having a corresponding first edge pixel as the second pixel to be used, and not determining, as the second pixel to be used, pixels having no corresponding first edge pixel,

2. 2. A computer program according to claim 1, comprising: A computer program, wherein the first determination process is executed using reduced target image data generated by performing a reduction process on the target image data, and reduced template image data generated by performing a reduction process on the template image data.

3. A computer program according to claim 1 or 2, a computer program product, wherein in the use pixel determination process, the plurality of first use pixels are determined from among pixels included in a partial image which is a part of the target image corresponding to the template image and which is determined based on the positional relationship determined in the first determination process.

4. 2. A computer program according to claim 1, comprising: A computer program, wherein the second edge pixel corresponding to a particular one of the first edge pixels is the second edge pixel that exists within a predetermined distance from the particular one of the first edge pixels in the positional relationship determined in the first determination process.

5. A computer program according to claim 1 or 4, a computer program product, in which, in the used pixel determination process, if the number of pixels among the plurality of first edge pixels included in a partial image which is a portion of the target image corresponding to the template image and which is determined based on the positional relationship determined in the first determination process is greater than the number of the second edge pixels, the first process is executed, and if the number of pixels among the plurality of first edge pixels included in a partial image which is a portion of the target image corresponding to the template image and which is determined based on the positional relationship determined in the first determination process is less than the number of the second edge pixels, the second process is executed.

6. A computer program according to any one of claims 1 to 5, The first determination process is a pattern matching process.

7. A computer program according to any one of claims 1 to 6, The second determination process is a process using an iterative closest point (ICP) algorithm.

8. An image processing device, an image acquisition unit that acquires target image data indicating a target image including an object and template image data indicating a template image corresponding to the object; an edge image specifying unit that specifies a plurality of first edge pixels in the target image using the target image data and specifies a plurality of second edge pixels in the template image using the template image data; a first determination unit that determines a relative positional relationship between the target image and the template image with a first accuracy; a pixel-to-be-used determination unit that determines a plurality of first use pixels from among the plurality of first edge pixels and determines a plurality of second use pixels from among the plurality of second edge pixels based on a positional relationship between the plurality of first edge pixels and the plurality of second edge pixels in the positional relationship determined by the first determination unit, wherein a sum of the plurality of first use pixels and the plurality of second use pixels is less than a sum of the plurality of first edge pixels and the plurality of second edge pixels; a second determination unit that determines a relative positional relationship between the target image and the template image with a second accuracy higher than the first accuracy, using the plurality of first use pixels and the plurality of second use pixels; Equipped with an image processing device, wherein the pixel to be used determination unit executes at least one of a first process, prior to processing by the second determination unit, in which, among the plurality of first edge pixels, pixels having a corresponding second edge pixel are determined as the first pixel to be used, and pixels having no corresponding second edge pixel are not determined as the first pixel to be used, and a second process, prior to processing by the second determination unit, in which, among the plurality of second edge pixels, pixels having a corresponding first edge pixel are determined as the second pixel to be used, and pixels having no corresponding first edge pixel are not determined as the second pixel to be used.

9. 1. An image processing method, comprising: an image acquiring step of acquiring target image data representing a target image including an object and template image data representing a template image corresponding to the object; an edge image identifying step of identifying a first plurality of edge pixels in the target image using the target image data and identifying a second plurality of edge pixels in the template image using the template image data; a first determination step of determining a relative positional relationship between the target image and the template image with a first accuracy; a pixel-to-be-used determining step of determining a plurality of first use pixels from among the plurality of first edge pixels and a plurality of second use pixels from among the plurality of second edge pixels based on a positional relationship between the plurality of first edge pixels and the plurality of second edge pixels in the positional relationship determined in the first determining step, wherein a sum of the plurality of first use pixels and the plurality of second use pixels is less than a sum of the plurality of first edge pixels and the plurality of second edge pixels; a second determination step of determining a relative positional relationship between the target image and the template image with a second accuracy higher than the first accuracy, using the plurality of first use pixels and the plurality of second use pixels; Equipped with an image processing method in which the use pixel determination step includes at least one of a first step, prior to the second determination step, of determining, among the plurality of first edge pixels, pixels having a corresponding second edge pixel as the first use pixel, and not determining, as the first use pixel, pixels having no corresponding second edge pixel, and a second step, prior to the second determination step, of determining, among the plurality of second edge pixels, pixels having a corresponding first edge pixel as the second use pixel, and not determining, as the second use pixel, pixels having no corresponding first edge pixel.

Images