IMAGE PROCESSING APPARATUS, IMAGE PROCESSING METHOD, AND IMAGE PROCESSING PROGRAM

Abstract

To provide an apparatus, system and method for image processing and a terminal device that allow for improved accuracy of pattern detection.SOLUTION: An apparatus for image processing 10 comprises a control section 12 that detects a work-piece concerned 31 suited to a template image 40 from an image concerned 30. The control section 12 acquires a partial template image 41, made by deleting a part of the template image 40, and sets up a restoration area 53 in which a lack area 52, deleted from the template image 40 in the partial template image 41, is combined with a partial extraction area 51 suited to the partial template image 41 in the image concerned 30. If a coincidence degree between a logical-product image 74 which a logical product is taken of a restoration image 63 for the restoration area 53 of the image concerned 30 and the template image 40 and the template image 40 is equal to or greater than a coincidence threshold, the control section 12 detects that in the image concerned 30 the work-piece concerned 31 is included in the restoration area 63.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to an image processing device, an image processing method, and an image processing program. [Background technology]

[0002] Conventionally, when detecting target workpieces of different colors, a method has been known that does not require color information from a higher-level control device, can reduce the memory capacity required for registering templates, and can reduce the detection time required for template matching (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 8-212346 Summary of the Invention [Problem to be solved by the invention]

[0004] When only the original template is targeted for detection, there is a risk of the detection accuracy being reduced if the shape of the extraction target has a defect.

[0005] In view of the above circumstances, an object of the present disclosure is to improve the pattern detection accuracy. [Means for solving the problem]

[0006] An image processing device according to an embodiment of the present disclosure includes a control unit that detects a target workpiece that matches a template image from a target image. The control unit acquires a partial template image in which a part of the template image is missing. The control unit sets a restoration area in which a missing area in the partial template image that is missing from the template image is combined with a partial extraction area in the target image that matches the partial template image. The control unit detects that the target workpiece is included in the restoration area in the target image when a logical product image obtained by logically multiplying a restoration image of the restoration area of ​​the target image and the template image has a degree of match between the template image and a match threshold value or more.

[0007] An image processing method according to an embodiment of the present disclosure includes an image processing device that detects a target workpiece that matches a template image from a target image, and obtains a partial template image in which a part of the template image is missing. The image processing method includes the image processing device setting a restoration area in which a missing area missing from the template image in the partial template image is combined with a partial extraction area in the target image that matches the partial template image. The image processing method includes the image processing device detecting that the target workpiece is included in the restoration area in the target image when a logical product image obtained by logically multiplying a restoration image of the restoration area of ​​the target image and the template image has a degree of agreement with the template image that is equal to or greater than a matching threshold.

[0008] An image processing program according to an embodiment of the present disclosure causes an image processing device that detects a target workpiece that matches a template image from a target image to acquire a partial template image in which a part of the template image is missing. The image processing program causes the image processing device to set a restoration area in which a missing area in the partial template image that is missing from the template image is combined with a partial extraction area in the target image that matches the partial template image. The image processing program causes the image processing device to detect that the target workpiece is included in the restoration area in the target image when a logical product image obtained by logically multiplying a restoration image of the restoration area of ​​the target image and the template image has a degree of coincidence between the template image and a coincidence threshold value or more. Effect of the Invention

[0009] According to an image processing device, an image processing method, and an image processing program according to an embodiment of the present disclosure, the pattern detection accuracy can be improved. [Brief description of the drawings]

[0010] [Figure 1] 1 is a block diagram showing an example of the configuration of an image processing system according to an embodiment. [Diagram 2] FIG. 2 is a diagram showing an example of a target image. [Diagram 3] FIG. 3 is an enlarged view of the area A enclosed by the dashed line in FIG. [Figure 4] FIG. 13 is a diagram illustrating an example of a template. [Figure 5A] FIG. 13 is a diagram showing an example in which the upper half of a template is used as a partial template. [Figure 5B] FIG. 13 is a diagram showing an example in which the lower half of a template is used as a partial template. [Figure 5C] FIG. 13 is a diagram showing an example in which the right half of a template is used as a partial template. [Figure 5D] FIG. 13 is a diagram showing an example in which the left half of a template is used as a partial template. [Figure 6]FIG. 13 is a diagram showing an example of a partial extraction region set in a target image. [Figure 7] FIG. 7 is an enlarged view of the area C enclosed by the dashed line in FIG. [Figure 8] 8 is a diagram showing an example of a restoration area set based on the partial extraction area of ​​FIG. 7; [Figure 9] 9 is a diagram showing the degree of match between a logical product image generated based on the restored region in FIG. 8 and a template image. [Figure 10] FIG. 7 is an enlarged view of the area D enclosed by the dashed line in FIG. [Figure 11] 11 is a diagram showing the degree of match between a logical product image generated based on a restoration area set from the partial extraction area of ​​FIG. 10 and a template image. FIG. [Figure 12] 1 is a flowchart illustrating an example of a procedure of an image processing method according to an embodiment. [Figure 13] FIG. 13 is a diagram showing an example of setting a partial template in a template. [Figure 14A] 4 is an enlarged view of the area B enclosed by the dashed line in FIG. 3, showing a state in which a non-target workpiece is overlapped on a target workpiece. FIG. [Figure 14B] 4 is an enlarged view of a portion B enclosed by a dashed line in FIG. 3, showing a state in which colors are mixed in an area where a target workpiece and a non-target workpiece overlap. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] (Example of configuration of image processing system 1) 1, an image processing system 1 according to an embodiment includes an image processing device 10 and an image acquisition device 20. The image processing device 10 acquires an image from the image acquisition device 20 or another device, and detects various patterns, such as characters, symbols, figures, or combinations of these, contained in the acquired image by template matching. Each component of the image processing system 1 will be described below.

[0012] <Image processing device 10> The image processing device 10 includes a control unit 12. The control unit 12 controls at least a part of the image processing device 10. The control unit 12 may include one or more processors. In this embodiment, the "processor" is a general-purpose processor, a dedicated processor specialized for a specific process, etc., but is not limited to these. The control unit 12 may include one or more dedicated circuits. The dedicated circuits may include, for example, an FPGA (Field-Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit). The control unit 12 may include a dedicated circuit instead of a processor, or may include a dedicated circuit together with a processor.

[0013] The image processing device 10 may further include a storage unit. The storage unit may be, for example, a semiconductor memory, a magnetic memory, an optical memory, or the like, but is not limited to these. The storage unit may function as, for example, a main storage device, an auxiliary storage device, or a cache memory. The storage unit may include an electromagnetic storage medium such as a magnetic disk. The storage unit may include a non-transitory computer-readable medium. The storage unit stores any information used in the operation of the image processing device 10. For example, the storage unit may store a system program, an application program, or the like. The storage unit may be included in the control unit 12.

[0014] The image processing device 10 may further include an interface for communicating with the image acquisition device 20 or other devices and outputting image processing results. The interface may include a communication module. The communication module may be compatible with mobile communication standards such as 4G (4th Generation) or 5G (5th Generation). The communication module may be compatible with communication standards such as LAN (Local Area Network). The communication module may be compatible with wired or wireless communication standards. The communication module may be compatible with various communication standards without being limited to these. The interface may include a display device that outputs visual information such as images, characters, or figures. The display device may include, for example, an LCD (Liquid Crystal Display), an organic EL (Electro-Luminescence) display or an inorganic EL display, or a PDP (Plasma Display Panel). The display device is not limited to these displays and may include displays of various other types. The display device may be included with a light-emitting device such as an LED (Light Emission Diode) or an LD (Laser Diode). The display device may be included with various other devices. The interface may include an input device that accepts input of information or data from a user. The input device may include, for example, a touch panel or a touch sensor, or a pointing device such as a mouse. The input device may include physical keys. The input device may include a voice input device such as a microphone.

[0015] <Image Acquisition Device 20> The image acquisition device 20 may be configured to include an imaging device such as a camera. The imaging device outputs an image obtained by capturing an image printed on paper, an image displayed on a display, or an image projected on a screen or a wall to the image processing device 10. The image processing device 10 may be configured to include a scanner. The scanner outputs an image obtained by scanning an image printed on paper to the image processing device 10. In other words, the image acquisition device 20 may output an image obtained by capturing or scanning a real image to the image processing device 10. An image acquired by the image acquisition device 20 by capturing or scanning a real image is also referred to as an acquired image. The image processing device 10 may acquire the acquired image of the image acquisition device 20 as the target image 30.

[0016] (Example of operation of image processing system 1) In the image processing system 1 according to this embodiment, the control unit 12 of the image processing device 10 detects various patterns consisting of characters, symbols, figures, or combinations of these from an image by template matching. Template matching compares a template prepared in advance with an image, and detects a part of the image that matches the template.

[0017] The control unit 12 detects a target workpiece 31 represented by an x ​​mark from within a target image 30 shown in Fig. 2, for example, by template matching. The target image 30 illustrated in Fig. 2 is a graph in which a plurality of points are plotted on a plane represented by an X axis and a Y axis. The target image 30 includes, in addition to the target workpiece 31, a non-target workpiece 32 represented by a rectangle and a non-target workpiece 33 represented by a diamond. The control unit 12 detects only the target workpiece 31 without detecting the non-target workpieces 32 and 33.

[0018] 3, even if there is a pattern in which the target work 31 and the non-target work 32 overlap, the control unit 12 detects the target work 31 that is overlapped by other work. In other words, the control unit 12 detects the target work 31 not only in a pattern in which the target work 31 exists alone, but also in a pattern in which the target work 31 overlaps with other work.

[0019] As illustrated in FIG. 4, the template includes a template region 50 and a template image 40 that identifies a pattern in the template region 50. The template image 40 represents a pattern of the target workpiece 31. The control unit 12 scans the target image 30 with the template and determines whether a region that matches the template image 40 exists in the target image 30. That is, the control unit 12 compares the target image 30 with the template image 40. If a region that matches the template image 40 exists, the control unit 12 detects the target workpiece 31 in that region. Here, the match between the template image 40 and the target image 30 means that the template image 40 and the target image 30 match at a rate of area equal to or greater than a predetermined value in the template region 50. The match between the template image 40 and the target image 30 includes a case where the template image 40 and the target image 30 match completely.

[0020] Here, the pattern illustrated in Fig. 3 is a pattern in which the target workpiece 31 and the non-target workpiece 32 overlap. Such a pattern does not match the template image 40 because it includes not only the cross mark but also a part of a rectangular pattern. Therefore, the control unit 12 cannot detect the target workpiece 31 in the pattern illustrated in Fig. 3 by simply comparing the target image 30 with the template image 40.

[0021] The image processing device 10 according to the present embodiment can detect a target workpiece 31 that is overlapped with another workpiece as illustrated in Fig. 3 by using a partial template in which a part of a template is missing. A specific detection procedure will be described below.

[0022] <Getting a partial template> The control unit 12 of the image processing device 10 acquires a partial template. As illustrated in FIG. 5A to FIG. 5D, the partial template includes a partial template region 51 and a partial template image 41 that specifies a pattern in the partial template region 51. The partial template is generated by missing a part of the template. Therefore, the partial template region 51 corresponds to a part of the template region 50. The partial template image 41 corresponds to an image of the template image 40 that specifies a pattern in the partial template region 51. The missing region of the template region 50 is called a missing region 52. In other words, the missing region 52 corresponds to an area that is missing from the template region 50 to generate the partial template region 51. The combined region of the partial template region 51 and the missing region 52 corresponds to the template region 50. The image that specifies the pattern in the missing region 52 is also called a missing image 42. In FIG. 5A to FIG. 5D, the missing image 42 is represented by a dashed line.

[0023] The partial template shown in Fig. 5A is configured such that the lower half of the template is missing, leaving the upper half as the partial template region 51. The partial template shown in Fig. 5B is configured such that the upper half of the template is missing, leaving the lower half as the partial template region 51. The partial template shown in Fig. 5C is configured such that the left half of the template is missing, leaving the right half as the partial template region 51. The partial template shown in Fig. 5D is configured such that the right half of the template is missing, leaving the left half as the partial template region 51.

[0024] <Pattern extraction using partial templates> The control unit 12 applies the partial template to the target image 30 to extract a region that matches the partial template image 41. For example, the control unit 12 may apply the partial template to the upper half of the template as the partial template region 51 shown in FIG. 5A to extract a pattern from the target image 30. The control unit 12 extracts a region of the partial template region 51 that matches the partial template image 41 and sets it as a partial extraction region 61. In FIG. 6, the upper half of the target workpiece 31 matches the partial template image 41. Therefore, the control unit 12 extracts the region of the upper half of the target workpiece 31 and sets it as the partial extraction region 61. In addition, the lower part of the non-target workpiece 33 matches the partial template image 41. Therefore, the control unit 12 extracts the region of the lower part of the non-target workpiece 33 and sets it as the partial extraction region 61.

[0025] In the above example, the control unit 12 applies the partial template shown in Fig. 5A to the target image 30, but this is not limited to this. The control unit 12 may apply the partial template shown in Fig. 5B, 5C, or 5D, or a partial template in which a template is missing in another manner, to the target image 30.

[0026] The target workpiece 31 that is not overlapped with other workpieces matches the template image 40 in Fig. 4. Therefore, the control unit 12 extracts the entire target workpiece 31 as an extraction region 60, and detects the target workpiece 31 from the target image 30.

[0027] <Determining whether the extracted pattern is part of the pattern of the target work 31> The control unit 12 determines whether the partial template image 41 extracted by applying the partial template to the target image 30 is an image of a part of the target workpiece 31 or an image of a part of another workpiece. In other words, the control unit 12 determines whether the pattern extracted by applying the partial template to the target image 30 is a part of the pattern of the target workpiece 31.

[0028] With reference to Fig. 7, which is an enlarged view of the dashed-line enclosed area represented by C in Fig. 6, a determination will be described in the case where a pattern in which another work overlaps a target work 31 is extracted using a partial template. The control unit 12 sets a missing region 52 corresponding to a partial extraction region 61 in the target image 30. The missing region 52 corresponds to a region that has been missing from the template region 50 in order to generate the partial template region 51.

[0029] As shown in FIG. 8, the control unit 12 extracts an image of the partial extraction region 61 from the target image 30 and generates it as a partial extraction image 71. The partial extraction image 71 corresponds to an image in which the missing region 52 is missing in the original target image 30. In order to complement the missing region in the partial extraction image 71, the control unit 12 sets a complementation region 62 corresponding to the missing region 52, extracts an image of the complementation region 62 from the target image 30, and generates it as a complementation image 72. The control unit 12 sets a restoration region 63 that combines the partial extraction region 61 and the complementation region 62. The control unit 12 generates a restoration image 73 of the restoration region 63 by combining the partial extraction image 71 of the partial extraction region 61 and the complementation image 72 of the complementation region 62. The restoration image 73 is an image that combines the partial extraction image 71 and the complementation image 72.

[0030] In other words, the control unit 12 may generate a region that combines the partial extraction region 61 and the missing region 52 in the image in which the target work 31 and the non-target work 32 overlap as illustrated in Fig. 7, and extract the image included in that region from the target image 30. The image extracted in this manner corresponds to the restored image 73.

[0031] 9, the control unit 12 generates a logical product image 74 by taking the logical product of the restored image 73 and the template image 40. The control unit 12 can generate the logical product image 74 as a set of pixels by taking the logical product of each pixel of the restored image 73 and the template image 40. The control unit 12 takes the logical product of each pixel of the restored image 73 and each pixel of the template image 40 such that the restored area 63 and the template area 50 overlap each other.

[0032] The control unit 12 calculates the degree of match between the logical product image 74 and the template image 40. The closer the logical product image 74 and the template image 40 are to a perfect match, the higher the degree of match is calculated to be. For example, the control unit 12 may compare the logical product image 74 and the template image 40 for each pixel and calculate the proportion of matching pixels as the degree of match.

[0033] When the degree of match between the logical product image 74 and the template image 40 is equal to or greater than a predetermined value, the control unit 12 determines that the restored image 73 includes the target workpiece 31, and detects the target workpiece 31 included in the restored area 63 of the target image 30. The predetermined value to be compared with the degree of match is also called a match threshold, and may be appropriately set to various values ​​such as 80%. In other words, the control unit 12 detects the target workpiece 31 included in the restored area 63 of the target image 30 when the degree of match is equal to or greater than the match threshold.

[0034] 6. The determination when the non-target work 33 is extracted by the partial template will be described with reference to FIG. 10, which is an enlarged view of the dashed-line enclosed area represented by D in FIG. 6. The control unit 12 sets a missing area 52 corresponding to the partial extraction area 61 in the target image 30. In FIG. 10, no pattern exists in the missing area 52. Therefore, the control unit 12 generates a restored image 73 including only the upper half pattern of the cross marks included in the partial extraction area 61 in the restored area 63 as shown in FIG. 11. The control unit 12 generates a logical product image 74 by performing a logical product of the restored image 73 and the template image 40. The logical product image 74 includes only the upper half pattern of the cross marks, similar to the restored image 73.

[0035] The control unit 12 calculates the degree of match between the logical product image 74 and the template image 40. The logical product image 74 matches only the pattern of the upper half of the template image 40. Therefore, the degree of match calculated in the example of FIG. 11 is lower than the degree of match calculated in the example of FIG. 9. If the degree of match between the logical product image 74 and the template image 40 is less than the match threshold, the control unit 12 determines that the restored image 73 does not include the target work 31. The control unit 12 may set the match threshold so that the degree of match calculated in the example of FIG. 9 is equal to or greater than the match threshold, and the degree of match calculated in the example of FIG. 11 is less than the match threshold.

[0036] As described above, the control unit 12 can extract the target work 31 even when it overlaps with another work by using the partial template. In addition, the control unit 12 does not detect a work that only partially matches the target work 31 as the target work 31 by generating the restored image 73 and the logical product image 74. In other words, the control unit 12 can improve the detection accuracy of the target work 31.

[0037] <Example of image processing procedure> The control unit 12 of the image processing device 10 may execute an image processing method including, for example, the procedure of the flowchart illustrated in Fig. 12. The image processing method may be realized as an image processing program executed by the control unit 12. The image processing program may be stored in a non-transitory computer-readable medium.

[0038] The control unit 12 acquires the target image 30 (step S1). The control unit 12 acquires a template (step S2). The control unit 12 acquires a partial template (step S3). The control unit 12 extracts a pattern that matches the partial template in the target image 30, and sets a partial extraction region 61 that includes the pattern (step S4). The control unit 12 sets a restoration region 63 that combines the partial extraction region 61 with a complement region 62 (step S5). The control unit 12 generates a logical product image 74 between a restoration image 73 of the restoration region 63 and the template image 40 of the template (step S6).

[0039] The control unit 12 calculates the degree of matching between the logical product image 74 and the template image 40 (step S7). The control unit 12 determines whether the degree of matching is equal to or greater than a predetermined threshold (step S8). If the degree of matching is equal to or greater than the predetermined threshold (step S8: YES), the control unit 12 detects that the target work 31 is included in the restoration area 63 in the target image 30. That is, the control unit 12 detects the target work 31 in the target image 30 (step S9). After executing the procedure of step S9, the control unit 12 ends the execution of the procedure of the flowchart in FIG. 12. If the degree of matching is not equal to or greater than the predetermined threshold (step S8: NO), that is, if the degree of matching is less than the predetermined threshold, the control unit 12 ends the execution of the procedure of the flowchart in FIG. 12.

[0040] (Small summary) As described above, according to the image processing device 10 and image processing method of this embodiment, by using a partial template, the target work 31 can be extracted even if it overlaps with another work. In this way, the presence of the target work 31 in the target image 30 is less likely to be overlooked. In addition, by generating the restored image 73 and the logical product image 74, a work that only partially matches the target work 31 is not detected as the target work 31. In this way, other work is less likely to be erroneously detected as the target work 31. As a result, the detection accuracy of the target work 31 is improved.

[0041] (Other embodiments) An example of the configuration of an image processing system 1 according to another embodiment will be described below.

[0042] <Partial template aspects> As exemplified in Fig. 13, a partial template may include a plurality of partial template regions 51 and a partial template image 41 that specifies a pattern within each partial template region 51. In the example of Fig. 13, the partial template regions 51 are located at four corners of the template, spaced apart from one another. The partial template regions 51 are not limited to this example, and may be located at both ends of the template, spaced apart from one another, for example. The number of partial template regions 51 is not limited to four, and may be three or less, or five or more.

[0043] When the partial template includes multiple partial template regions 51, the control unit 12 acquires a first partial template image and a second partial template image as the partial template image 41. When the control unit 12 can extract a region that matches the first partial template image from the target image 30 and can extract a region that matches the second partial template image from the target image 30, the control unit 12 may detect that the target workpiece 31 is included in the restoration region 63 in the target image 30.

[0044] When the partial template includes one or more partial template regions 51, the control unit 12 calculates the sum of the areas of the parts that match the partial template image 41 and the sum of the areas of the non-matching parts in the restoration region 63. When the area of ​​the matching parts is equal to or greater than the area of ​​the non-matching parts, the control unit 12 may detect that the target workpiece 31 in the target image 30 is included in the restoration region 63. Conversely, when the area of ​​the matching parts is less than the area of ​​the non-matching parts, the control unit 12 may determine that the target workpiece 31 in the target image 30 is not included in the restoration region 63 even if some regions that match the partial templates can be extracted.

[0045] <Calculation of the coordinates of the extracted target work 31> The control unit 12 may set a coordinate system in the target image 30. For example, the control unit 12 may determine the X-axis and Y-axis included in the target image 30 in the graph of Fig. 2, and set an XY coordinate system in the target image 30. When the control unit 12 detects a target workpiece 31 in the target image 30, it may calculate the coordinates of the target workpiece 31 in the set coordinate system. By calculating the coordinates of the target workpiece 31, the control unit 12 may convert the graph acquired as the target image 30 into numerical data and output it.

[0046] The control unit 12 may, for example, determine a feature point of the detected target workpiece 31 and calculate the coordinates of the feature point as the coordinates of the target workpiece 31. The feature point may be the center of the pattern of the target workpiece 31 or a corner of the pattern. For example, if the target workpiece 31 is an x ​​mark, the feature point may be an intersection of the x mark or an end of the line that constitutes the x mark.

[0047] <Extraction of target work 31 taking color information into consideration> The control unit 12 may generate a logical product image 74 based on color information of the restored image 73 of the restored area 63 set in the target image 30 and color information of the template image 40. For example, if the color of the pattern of the target work 31 is red, the control unit 12 may add red color information in the logical product image 74 to pixels that are red in both the restored image 73 and the template image 40. In addition, if the color of another work different from the target work 31 is blue, the control unit 12 does not generate a pattern of that part in the logical product image 74, even if there is a part where the other work included in the restored image 73 overlaps with the pattern of the template image 40.

[0048] When the restored image 73 includes a pattern in which the target work 31 overlaps with other workpieces, the control unit 12 can distinguish the target work 31 from the other workpieces based on the difference in color between the target work 31 and the other workpieces.

[0049] For example, as shown in FIG. 14A, the target work 31 and the non-target work 32 (other work) are included in the restored image 73. In FIG. 14A, the line pattern of the non-target work 32 is superimposed on the line pattern of the target work 31. The lines of the target work 31 are divided by the lines of the non-target work 32. In this case, the logical product image 74 generated by taking the logical product of the restored image 73 and the template image 40 includes a pattern in which the lines are divided at the portion where the non-target work 32 overlaps with the target work 31. That is, the pattern corresponding to the target work 31 in the logical product image 74 may be missing due to the non-target work 32. The degree of match calculated based on the logical product image 74 with a missing pattern and the template image 40 may be lower than the actual degree of match. The calculated degree of match becomes lower, which increases the possibility that the target work 31 that should be detected will be overlooked.

[0050] The control unit 12 may extract a portion of the target work 31 where the pattern of the target work 31 is missing due to the non-target work 32 overlapping the target work 31 from the restored image 73 based on the color information of the target work 31 and the color information of the non-target work 32. The control unit 12 may add the extracted portion to the pattern of the logical product image 74. In this way, the pattern missing in the target work 31 due to the non-target work 32 is repaired. By repairing the missing pattern, the degree of match calculated based on the logical product image 74 and the template image 40 is brought closer to the actual degree of match. As a result, the target work 31 that should be detected is less likely to be overlooked.

[0051] For example, as shown in FIG. 14B, the color of the intersection 35 between the line of the target work 31 and the line of the non-target work 32 may be a mixture of the color of the target work 31 and the color of the non-target work 32. In this case, the logical product image 74 generated by performing a logical product of the restored image 73 and the template image 40 includes a pattern in which the line of the target work 31 is divided at the intersection 35. That is, the pattern corresponding to the target work 31 in the logical product image 74 is missing at the intersection 35. The control unit 12 may extract a portion (intersection 35) in the restored image 73 in which the color of the target work 31 and the color of the non-target work 32 are mixed. The control unit 12 may add the extracted portion to the pattern of the logical product image 74. In this way, the pattern missing in the target work 31 due to the non-target work 32 is repaired. By repairing the missing pattern, the degree of match calculated based on the logical product image 74 and the template image 40 is brought closer to the actual degree of match. As a result, the target workpiece 31 that should be detected is less likely to be overlooked.

[0052] <Generating partial templates> The control unit 12 may generate a partial template based on the template. For example, when the template image 40 is a cross mark, the control unit 12 may extract the intersections of the cross marks as feature points, and generate regions obtained by dividing the template region 50 by lines passing through the intersections of the cross marks as partial template regions 51. When the template image 40 is a circle mark, the control unit 12 may extract the center of the circle as a feature point, and generate regions obtained by dividing the template region 50 by lines passing through the center of the circle as partial template regions 51. In other words, the control unit 12 may extract feature points of the template image 40, and generate the partial template region 51 and the partial template image 41 based on the feature points.

[0053] <Detection of target workpiece 31 from acquired image> The control unit 12 detects the target workpiece 31 using the acquired image obtained by photographing or scanning a real image by the image acquisition device 20 as the target image 30. When the image acquisition device 20 photographs or scans a real image, the real image may be distorted and reflected in the acquired image due to distortion of the optical system of the image acquisition device 20. The control unit 12 may set a partial extraction region 61 that fits the partial template image 41 in the target image 30 based on the distortion of the optical system of the image acquisition device 20. For example, the optical system of the image acquisition device 20 may have an optical axis that is inclined at a predetermined angle with respect to the normal direction of the surface on which the real image is displayed or projected. In this case, the control unit 12 may set the partial extraction region 61 that fits the partial template image 41 based on the predetermined angle, taking into account the distortion of the pattern reflected in the acquired image as the target image 30.

[0054] The real image of the object photographed or scanned by the image acquisition device 20 may itself be distorted. For example, an image photographed of an image displayed or projected on a surface such as a wall having at least one of a concave portion or a convex portion may differ from the image originally displayed or projected due to the influence of the concave portion or the convex portion. Also, an image photographed or scanned of an image printed on a booklet or the like may differ from the image originally printed on the booklet or the like due to the influence of the paper surface being curved when the booklet is opened or the influence of the deformation of the paper surface such as wrinkles in the paper. In this case, the control unit 12 may set a partial extraction region 61 that fits the partial template image 41, taking into consideration the distortion of the pattern reflected in the acquired image as the object image 30, based on the influence of the concave portion or the convex portion on the surface on which the real image is projected or printed on the acquired image.

[0055] As described above, the control unit 12 can detect the target workpiece 31 by taking into consideration the distortion or deformation of the acquired image as the target image 30. As a result, the detection accuracy of the target workpiece 31 can be improved.

[0056] Although the embodiments of the present disclosure have been described based on the drawings and examples, it should be noted that those skilled in the art can make various modifications and alterations based on the present disclosure. Therefore, it should be noted that these modifications and alterations are included in the scope of the present disclosure. For example, the functions included in each means or step can be rearranged so as not to be logically inconsistent, and multiple means or steps can be combined or divided into one. [Explanation of symbols]

[0057] 1. Image Processing System 10 Image processing device (12: control unit) 20 Image acquisition device 30 Target image (31: Target work, 32, 33: Non-target work, 35: Intersection) 40 template images (41: partial template image, 42: missing image) 50 template region (51: partial template region, 52: missing region) 60 Extraction area 61 Partial extraction area 62 Complementary area 63 Restore Area 71 Partially Extracted Images 72 Complementary Image 73 Reconstructed image 74 Logical Product Images

Claims

1. A control unit is provided to detect a target workpiece that matches the template image from the target image, The control unit is A partial template image is obtained by removing a part of the template image; a restoration area is set by combining a missing area, which is missing from the template image in the partial template image, with a partial extraction area in the target image that matches the partial template image; An image processing device that detects that the target work is included in the restoration area in the target image when a degree of match between a logical product image obtained by logically multiplying a restored image of the restoration area of ​​the target image and the template image and the template image is equal to or greater than a matching threshold.

2. The image processing device according to claim 1 , wherein the control unit extracts an image in the target image that matches the partial template image as a partial extraction image, and sets an area in the target image in which the partial extraction image exists as the partial extraction area.

3. The image processing device according to claim 1 or 2, wherein the control unit sets a coordinate system in the target image and calculates coordinates of feature points of the target work detected in the target image as coordinates of the target work.

4. The image processing device according to claim 1 , wherein the control unit generates the logical product image based on color information of the restored image and color information of the template image.

5. The image processing device described in claim 4, wherein the control unit extracts a portion of the restored image where the pattern of the target work is missing due to the other work overlapping the target work based on color information of the target work and color information of the other work, and adds the extracted portion to the pattern of the logical product image.

6. The image processing device according to claim 4 or 5, wherein the control unit extracts a portion of the restored image where the color of the target work is a mixture of the color of the target work and the color of another work, and adds the extracted portion to the pattern of the logical product image.

7. The control unit is A first partial template image and a second partial template image are obtained as the partial template images; 7. An image processing device as claimed in claim 1, which detects that the target work is included in the restoration area in the target image when both a portion that matches the first partial template image and a portion that matches the second partial template image are included in the restoration area.

8. 8. The image processing device according to claim 1, wherein the control unit detects that the target work is included in the restoration area in the target image when an area of ​​a portion in the restoration area that matches the partial template image is equal to or greater than an area of ​​a portion that does not match the partial template image.

9. The image processing device according to claim 1 , wherein the control unit extracts feature points of the template image, and generates the partial template image based on the feature points.

10. The image processing device according to claim 1 , wherein the control unit acquires, as the target image, an acquired image output by an image acquisition device that captures a real image.

11. the image acquisition device captures the real image using an optical system having an optical axis that is inclined at a predetermined angle with respect to a normal direction of a surface on which the real image is displayed or projected; The image processing device according to claim 10 , wherein the control unit sets the partial extraction region that matches the partial template image in the target image based on the predetermined angle.

12. the image acquisition device captures an image displayed or projected on a surface having at least one of a concave portion and a convex portion, and outputs the captured image; The image processing device according to claim 10 , wherein the control unit sets the partial extraction region that matches the partial template image in the target image based on an effect that the concave portion or the convex portion has on the target image.

13. An image processing device that detects a target workpiece that matches a template image from a target image acquires a partial template image by missing a part of the template image; the image processing device sets a restoration area by combining a missing area, which is missing from the template image in the partial template image, with a partial extraction area, which matches the partial template image, in the target image; The image processing method includes the image processing device detecting that the target work is included in the restoration area in the target image when the degree of match between a logical product image obtained by logically multiplying the restored image of the restoration area of ​​the target image and the template image is equal to or greater than a matching threshold.

14. The image processing method according to claim 13, further comprising the step of: extracting an image in the target image that matches the partial template image as a partial extraction image; and setting an area in the target image in which the partial extraction image exists as the partial extraction area.

15. The image processing method according to claim 13 or 14, further comprising the image processing device setting a coordinate system in the target image and calculating the coordinates of the feature points of the target work detected in the target image as the coordinates of the target work.

16. The image processing method according to claim 13 , further comprising: generating the logical product image based on color information of the restored image and color information of the template image, the image processing device generating the logical product image based on color information of the restored image and color information of the template image.

17. An image processing device that detects a target workpiece that matches a template image from a target image acquires a partial template image by removing a part of the template image; causing the image processing device to set a restoration area in which a missing area that is missing from the template image in the partial template image is combined with a partial extraction area that matches the partial template image in the target image; An image processing program that causes the image processing device to detect that the target work is included in the restoration area in the target image when a degree of match between a logical product image obtained by logically multiplying a restored image of the restoration area of ​​the target image and the template image is equal to or greater than a matching threshold.

18. 18. The image processing program according to claim 17, further comprising causing the image processing device to extract an image in the target image that matches the partial template image as a partial extraction image, and to set an area in the target image in which the partial extraction image exists as the partial extraction area.

19. The image processing program according to claim 17 or 18, which causes the image processing device to set a coordinate system in the target image and calculate the coordinates of the feature points of the target work detected in the target image as the coordinates of the target work.

20. 20. The image processing program according to claim 17, further comprising causing the image processing device to generate the logical product image based on color information of the restored image and color information of the template image.

Images