Systems and methods for finding patterns in images using a vision system and classifying the patterns

Abstract

The present invention provides a system and method for finding patterns in images, which combines a neural network classifier. A pattern finding tool is coupled with the classifier, which can run before or after the tool to have a sub-pixel accurate marked pattern result. For a pattern finding tool that can detect multiple templates, its performance is improved when the neural network classifier notifies the pattern finding tool to work only on a subset of the templates initially trained. Similarly, initially the pattern finding tool detects a pattern, and then the neural network classifier can determine whether it has found the correct pattern. The neural network can also reconstruct / clean the shape of the imaging, and / or eliminate pixels that are not very relevant to the shape of interest, thus reducing the search time and significantly increasing the chance of locking the correct shape.

Description

Technical Field

[0001] The present invention relates to machine vision systems and methods, and more particularly, to pattern search and recognition tools. Background Art

[0002] Machine vision systems, also referred to herein as "vision systems", are used to perform various tasks in a manufacturing environment. Generally, a vision system consists of one or more cameras with image sensors (or "imagers") that acquire grayscale or color images of a scene containing an object being manufactured. The images of the object can be analyzed to provide data / information to a user and related manufacturing processes. The data generated by the images is typically analyzed and processed by a vision system in one or more vision system processors, which can be specially constructed or part of one or more software applications instantiated in a general-purpose computer (e.g., a PC, laptop, tablet, or smartphone).

[0003] Common vision system tasks include alignment and inspection. In an alignment task, a vision system tool (e.g., a well-known system purchased from Cognex Corporation of Massachusetts) compares features in a scene image with a trained (using actual or synthetic models) pattern and determines the presence / absence and pose of the pattern in the imaged scene. This information can be used for subsequent inspection (or other) operations to search for defects and / or perform other operations, such as part rejection.

[0004] There is a desire to improve the performance of traditional pattern finding tools, which may include a list of predefined searchable patterns (e.g., circles, squares, spiral images, etc.). Typically, such tools may not be able to correctly distinguish between certain trained patterns with subtle differences (e.g., a circle versus a circle with a small notch). Summary of the Invention

[0005] The present invention overcomes the disadvantages of the prior art by providing a system and method for finding patterns in images that incorporates a neural network (also referred to as a "neural net") classifier (sometimes also referred to as an "analyzer"). The pattern finding tool is coupled to the classifier, which can run before or after the tool to obtain a marked pattern result with sub-pixel accuracy. In the case of a pattern finding tool that can detect multiple templates, its performance is improved when the neural network classifier notifies the pattern finding tool to work only on a subset of the templates on which it was initially trained. Alternatively, the performance of the pattern finding tool can be improved when a neural network is used to reconstruct or clean the image before running the pattern finding tool. Additionally, the neural network can be used to calculate a weighted value for each pixel in the image based on the likelihood that the pixel belongs to the pattern to be matched. Similarly, initially the pattern finding tool detects a pattern, and then the neural network classifier can determine whether it has found the correct pattern.

[0006] In an illustrative embodiment, a system and method for finding patterns in images includes a pattern finding tool that is trained based on one or more templates associated with one or more training images that contain the pattern of interest. A neural network classifier is trained on the one or more training images, and at runtime a runtime template matching process is performed, where (a) the trained neural network classifier provides one or more templates to the pattern finding tool based on the runtime image, and the trained pattern finding tool performs pattern matching based on the one or more template images combined with the runtime image, or (b) the trained pattern finding tool provides the pattern found from the runtime image to the trained neural network classifier, and the trained neural network classifier performs pattern matching based on the found pattern and the runtime image. The pattern finding tool is adapted to be trained using multiple templates or on a single template. The neural network includes a convolutional neural network (CNN).

[0007] In another embodiment, a system and method for finding patterns in an image are provided. The system and method include a neural network that is trained to locate one or more candidate shapes in an image and is arranged to identify the probability that one or more shapes are present in the image during runtime operation. The neural network thus generates (a) a weighted mask of the features of one or more candidate shapes having a probability exceeding a probability threshold, and / or (b) a reconstructed image in which the features of the model replacing the one or more candidate shapes are present, wherein a neural network analyzer identifies the presence of the features of one or more candidate shapes exceeding the probability threshold. Illustratively, one or more models relative to the one or more candidate shapes are used to train a pattern finding tool to find the one or more candidate shapes in (a) the weighted mask and / or (b) the reconstructed image. The neural network can define the weighted mask such that each pixel therein has a score related to the identification of the one or more shapes. The reconstructed image can be defined as a binary image. Illustratively, the neural network analyzer provides data to the pattern finding tool regarding the presence of the type of one or more candidate shapes, and the pattern finding tool restricts the processing to that related to locating that type. Generally, the neural network can include a convolutional neural network (CNN). BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The following description of the invention refers to the accompanying drawings, in which:

[0009] Figure 1 is a schematic diagram of a vision system arranged to analyze an imaging object containing different shapes and / or patterns using a pattern finding tool in combination with a neural network classifier;

[0010] Figure 2 is a block diagram showing the training process of an intelligent pattern finding tool including a neural network classifier and a pattern finding tool trained on the same set of one or more image templates;

[0011] Figure 3 is for use Figure 2 of the trained intelligent pattern finding tool for runtime operation on an input image, wherein the neural network classifier operates before the pattern finding tool;

[0012] Figure 4 is a block diagram showing the training process of an intelligent pattern finding tool including a neural network classifier and a pattern finding tool trained on one or more images of a single template;

[0013] Figure 5 is for use Figure 4 of the trained intelligent pattern finding tool for runtime operation on an input image, wherein the neural network classifier operates after the pattern finding tool;

[0014] Figure 6 A flowchart showing an overview of a process of initially searching / determining candidate shapes in an image using a neural network and then applying a pattern finding tool to refine the search / determination;

[0015] Figure 7 To show Figure 6 A flowchart of the training and runtime operations of a neural network during the

[0016] Figure 8 To show the runtime operations of a pattern finding tool using the image results provided by the neural network process of Figure 7 A flowchart;

[0017] Figure 9 To show Figure 5 A block diagram of the creation of a weighted mask related to the pattern finding tool of

[0018] Figure 10 To show Figure 9 A schematic diagram of processing exemplary shapes of interest and image data containing the shapes of interest and a model of the shape of interest to generate a weighted mask according to

[0019] Figure 11 To show Figure 5 A block diagram of reconstructing or cleaning the shape of interest in the acquired image data using a neural network according to

[0020] Figure 12 To show a schematic diagram of processing exemplary acquired image data containing the shape of interest to generate a reconstructed and / or cleaned version of the shape according to Figure 11 (e.g., as a binary image); Detailed Description

[0021] I. System Overview

[0022] Figure 1FIG. 0 shows a general vision system apparatus 100 for use in accordance with illustrative systems and methods. The vision system may be implemented in any acceptable environment, including part / surface inspection, robot control, part alignment, etc. The system includes at least one vision system camera assembly 110 having optics O and an image sensor (also referred to as a “sensor” or “imager”) S, which may also include on-board illumination or stand-alone illumination (not shown). The camera assembly images a scene 120, which may include one or more stationary or moving objects 130. In this example, the objects include contoured shapes, as well as various internal shapes 132, 134, 136, and 138 of regular and irregular geometries. More generally, the pattern may be any 2-D image of any geometry or object.

[0023] The camera assembly 110 and associated sensor S are interconnected to a vision system processor 140, which may be located in whole or in part within the camera assembly 110, or may be located in a separate processing device such as a server, PC, laptop, tablet, or smartphone (computer 160). The computing device may include suitable user interactions such as a display / touchscreen 162, keyboard 164, and mouse 166.

[0024] Illustratively, the vision system processing (processor) 140 operates various vision system tools and associated software / firmware to manipulate and analyze the acquired and / or stored images of the object 130 during run time. The processing (processor) 140 may be trained to act according to specific parameters and uses a training process to identify specific shapes found in the object. The processing (processor) 140 includes various vision system components, including a pattern finding tool 142, such as those found in the software package and its variants (e.g., Multi-Model). The pattern finding tool may employ a training pattern or a standard shape pattern (square, circle, etc.) contained in a training template 144. As described below, the vision system processing (processor) also includes a neural network processing (processor) 150, or interfaces with the neural network processing (processor). The neural network processing (processor) (also referred to as a “neural network”) 150 operates on various patterns in the form of a classifier to improve the pattern finding speed and performance of the system 100.

[0025] The results of the pattern finding may be communicated to the user via a computer interface 162, and / or to another downstream use device or processing (processor) 180. Such a device or processing (processor) may include an assembly robot controller, in-line inspection, part inspection / rejection, quality control, etc.

[0026] It has been recognized that traditional pattern matching methods require the use of a model image with a shape or feature of interest to train a conventional pattern matching tool, such as or MultiModel. During run time, the pattern matching tool runs through one or more (possibly all) of the training templates in an effort to locate the correct match to the training pattern in the acquired image of the inspected object.

[0027] In contrast, the present embodiment provides an intelligent pattern finder tool that uses neural network processing to enhance a traditional pattern finder tool, thereby enabling it to automatically label the found pattern in the tool results or use a related neural network classifier to reliably detect the pattern. In operation, this method allows the intelligent pattern finder tool to be trained on an image database containing templates. After training, during run time, the intelligent pattern finder tool combines the best features of the traditional pattern finder and the neural network classifier to provide a correctly labeled pattern finding result with a highly accurate pose (position, scale, rotation, etc.).

[0028] II. Pattern Finding with Neural Network Optimization for Search

[0029] Refer to Figure 2 , which shows a block diagram representing the training time process 200 before run time. The tool 210 includes a conventional pattern finding tool 220 and a related neural network classifier that helps the pattern finding tool find the correct pattern in a set / multiple training patterns (templates) 240.

[0030] More specifically, at training time, the traditional pattern finding tool 220 (e.g., Multi-Model) is trained on one or more template images. At the same time, the neural network classifier (e.g., a convolutional neural network (CNN)) 230 is trained on multiple example images of the pattern represented by each template. The neural network classifier 230 is trained to process the input image and report a subset of the template labels found in the input image.

[0031] Figure 3 Depicts the run time process (using pre-classification) 300, where the trained neural network classifier 310 first runs on the input run time image (acquired by the camera 110 and / or stored through previous acquisition) 320 and determines the probability of each template. The intelligent pattern finding tool filters out the best results and then notifies the trained pattern finding tool 350 to process a subset of these M matched templates 340, rather than the complete set of N templates ( Figure 2Among them (240). In this way, a subset of the M best-fit templates 340 is provided as search parameters to the training pattern search tool 350. The input image 320 is provided to the pattern search tool 350, and the search parameters are used to generate a matching template result 360 as the output of the tool 350. These results can be displayed to the user or used for downstream usage operations (such as part calibration, inspection, etc.). Advantageously, the process 300 reduces the computational amount usually required to generate matching templates.

[0032] According to the embodiments herein, various proprietary and commercially available (e.g., open-source) neural network architectures and related classifiers can be employed. For example, TensorFlow, Microsoft CNTK.

[0033] An exemplary application where the above training process and runtime process 200 and 300 can be used is to find the correct fiducial, where the fiducial shapes of different parts can be different (cross, diamond, etc.). Illustratively, traditional pattern search tools are trained on template images representing each possible fiducial pattern. In addition, a neural network classifier (e.g., TensorFlow) is trained on multiple images that show the appearance variations of each fiducial pattern and the labels associated with each fiducial pattern. At runtime, first, the trained neural network classifier is run, which returns a set of labels found in the runtime image. Using this information, the system can notify the pattern search tool (e.g., MultiModel) to run only on the set of templates represented by the labels generated by the neural network classifier, thus accelerating alignment and generating more reliable results.

[0034] Figure 4 An apparatus is shown where a traditional pattern search tool searches for a certain pattern, and a neural network classifier determines whether it is the correct match (i.e., post-classification). At training time, the traditional pattern search tool 410 is trained with a single image template 440. Then, a neural network classifier (e.g., TensorFlow) 430 associated with the tool 410 is trained on multiple images for the desired template. The input to the classifier 430 is the same as the image input to the traditional pattern search tool 440.

[0035] Figure 5 A runtime process (using post-classification) 500 is depicted, where the pattern search tool 550 first searches for a pattern in the input image 520. The input image 520, along with an optional bounding box calculated from the output of the traditional pattern search tool (e.g., ) is provided to the trained neural network classifier 510. Then, the classifier 510 determines whether the traditional pattern finder has found the correct / matching pattern 560. The output of the classifier 510 is the overall confidence of finding the training template.

[0036] For example, an operational application example involves highly confusing shapes with minor differences, such as a circle and a notched circle. Suppose a traditional pattern finding tool (e.g., ) 350 is trained on a template image depicting a notched circle. Then, the neural network classifier 510 is trained on images that contain the desired shape (notched circle) as well as other confusing shapes (non-notched circles). At runtime, the input image and an optional bounding box calculated from the output of the traditional pattern finding tool are fed into the trained neural network classifier 510, and then the classifier determines whether the traditional pattern finder has found the correct pattern (notched circle). In this exemplary case, the process improves the robustness of pattern finding.

[0037] Note that in alternative embodiments, the traditional pattern finding tool and its ability to be trained on one or more templates are highly variable. In alternative embodiments, the above pre-classification and post-classification processes can each be modified to include different types of pattern finding tools and related templates.

[0038] III. Pattern Finding with Optimized Search Using a Trained Pattern Tool

[0039] Refer to Figure 6 , which shows the overall (general) process 600 of intelligent pattern finding according to another exemplary embodiment that can be implemented by the device 100 of Figure 1 . It is expected that it may be more challenging to locate some patterns using a conventionally trained pattern finding tool that operates on the acquired images. In some cases, the inherent characteristics of a neural network / deep learning architecture can provide benefits when initially locating pattern candidates in an image. Therefore, in process 600, the neural network is trained to locate various pattern types and is applied to the acquired image in step 610. This will generate a list of candidates with associated scores for a given type of shape. Then, based on the scores, process 600 applies a conventional pattern finding tool (e.g., MultiModel) to the shape candidates with scores higher than a specific threshold (step 620). The pattern finding tool searches for the specific shape identified by the neural network, or it can search for various types of shapes in each shape candidate.

[0040] Advantageously, the neural network can effectively identify possible shape candidates, while computationally intensive tasks, such as sub-pixel model fitting, can be handled in a robust manner by the pattern finding tool.

[0041] In Figure 7The training of a neural network to recognize certain shapes is described in step 710 of process 700. Once trained, during run time, the neural network assigns a score (probability) to each pixel in the acquired image based on whether it appears to be part of the trained shape using the trained configuration (step 720). The result is a probability image where each pixel in the image has an assigned score (step 730). The probability image from step 730 can be stored and then provided (e.g., as a mask) to a pattern finding tool - to mask out pixels that do not appear to have a candidate shape from the image results on which the pattern finding tool operates (step 740). The neural network results can include the type of candidate shape in the probability image. The shape type information allows the pattern finding tool to specifically narrow its search in the probability image (at selected locations) to the shape types provided by these results (step 750). Thus, the pattern finding tool can operate faster and more efficiently as it avoids running tools that are not relevant to the candidate shapes.

[0042] The above-described process 600 is advantageous in a variety of applications. For example, in the presence of high local distortion, it is useful to use a neural network to initially screen an image because the neural network essentially reconstructs the image based on probability in a more direct way for analysis by a pattern finding tool. For example, the incoming image could be highly textured and lack defined contrast lines. After being processed by the neural network, the resulting probability image is a binary representation with high contrast, with defined boundaries representing (e.g.) rectangles, triangles, circles, etc. In a specific example, the neural network can effectively solve the problem of the end shape of a rope or cable that may be worn (resulting in a highly textured area). The neural network passes a bright rectangle on a dark background to the pattern finding tool - and vice versa.

[0043] As Figure 8 As described in process 800, during run time, a trained pattern finding tool (already trained with a model-based template related to the shape of interest) receives the probability image (mask) from the neural network and (optionally) receives information about the type of candidate shape identified in the image (step 810). The pattern finding tool operates on the image, focusing on the selected area, and uses tools and processes related to the identified image type (step 820). Then, in step 830, the pattern finding tool generates results where the found shape is located within the image, and appropriate coordinate (and other) data on the shape is output for subsequent operations.

[0044] Further reference Figure 9, which shows a block diagram of an exemplary process 900 for creating and using a weighted mask related to a shape of interest in an acquired image. As shown, an image 910 is input into a neural network 920. Using appropriate training techniques, the neural network 920 outputs a weighted mask 930 of the shape of interest. As described above, each pixel is scored based on its likelihood of being part of the shape of interest. Subsequently, the weighted mask 930 and the original image data 910 are input into a pattern finding (template matching) tool (such as Cognex etc.). The tool 940 can thereby output the position of the shape of interest within the image 950 and additional matching score information 960 based on the data contained in the weighted mask 930.

[0045] Figure 10 The process 900 is depicted in the diagram 1000. The exemplary shape of interest 1010 is shown as a U-shaped structure with a continuous boundary. However, the related acquired image 1012 provides a discontinuous boundary 1014 and intermediate shapes 1016. Additionally, the shape of interest in the image 1012 is rotated by an angle within the scene relative to the expected shape 1010. There may also be other distortion-based differences between the acquired image and the expected shape. As described herein, the shape data 1010 of interest and the image data 1012 are input into a neural network 1020. The output weighted mask 1030 of the resulting image is represented as a series of shape segments 1040 that approximate the underlying shape of interest and omit the intermediate shape data 1016. As shown, the segments 1040 enclose a range of surrounding pixels with a higher probability / likelihood. This region approximates the general contour of the edge of the shape of interest. This representation 1040 is more easily matched by a conventionally model-trained pattern finding (template matching) tool.

[0046] In another exemplary embodiment, a neural network can be used to reconstruct and / or clean up the shape of interest within an image. As Figure 11 shown in the process 1100, a neural network 1120 receives the acquired image data 1110 and employs training to output a reconstruction 1130 of the shape of interest, where each pixel is scored based on its likelihood of belonging to the shape of interest (i.e., the object of neural network training). This reconstruction is then input into a model-based pattern finding (template matching) tool 1140 that includes a template of the shape of interest. The tool outputs a rough position 1150 of the shape of interest. This rough position can be appropriately used by a downstream process and / or optionally input again into a model training pattern finding tool 1160 (the same tool as block 1140 or a different tool). The original image data 1110 is also provided to the pattern finding tool 1160. The output of the tool 1160 from the inputs 1110 and 1150 is a fine position 1170 of the shape of interest within the image 1110.

[0047] As an example of process 1100, Figure 12 Diagram 1200 of shows two input shapes 1210 and 1212 in the image. As described above, neural network reconstructions 1220 and 1222 are performed on each shape. This results in reconstructed shapes 1230 and 1232 for the image data being acquired. The reconstruction can thus replace existing distorted or unclear shapes. Thus, the neural network can be used to effectively provide the clearing and / or reconstruction of incomplete or distorted shapes in the image data, which can allow for more effective use of such data in downstream operations, including pattern finding using the pattern finding tool described above or another suitable tool. As shown, the shape can be represented as a well-defined binary image having boundaries commensurate with the boundaries of the expected / model shape.

[0048] IV. Conclusion

[0049] It should be clear that the above systems and methods, using a combination of traditional pattern matching applications and neural network classifiers, provide a more reliable and faster technique for finding and matching training patterns. This approach allows for a reduction in the number of templates or the filtering of the found patterns, thus enhancing the system's and method's decision on the correct match. In addition, the above systems and methods effectively enable the neural network to be used as an imaging shape reconstruction / cleaning tool and / or eliminate pixels that are less relevant to the shape of interest, thus reducing the search time and significantly increasing the chance of locking onto the correct shape. This technique is particularly effective when the shapes in the image are distorted or lack shape features.

[0050] Exemplary embodiments of the present invention have been described in detail above. Various modifications and additions can be made without departing from the technical solutions and scope of the present invention. In order to provide various combinations of features in related new embodiments, the features of each of the above-described various embodiments can be appropriately combined with the features of other described embodiments. In addition, although multiple separate embodiments of the devices and methods of the present invention have been described above, what is described herein is merely an illustration of the application of the principles of the present invention. For example, as used herein, the terms "processing" and / or "processor" should be broadly understood to include various functions and components based on electronic hardware and / or software (and can alternatively be referred to as functional "modules" or "elements"). In addition, the depicted processes or processors can be combined with other processes and / or processors, or divided into various sub-processes or processors. According to the embodiments herein, such sub-processes and / or sub-processors can be combined differently. Similarly, it can be clearly envisioned that any function, process, and / or processor herein can be implemented using electronic hardware, software consisting of non-transitory computer-readable media of program instructions, or a combination of hardware and software. Additionally, as used herein, various directional and setting terms, such as "vertical", "horizontal", "upward", "downward", "bottom", "top", "side", "front", "rear", "left", "right", etc., are used only as relative conventions and not as absolute directions / settings relative to a fixed coordinate space (such as the direction of the action of gravity). Additionally, when the terms "substantially" or "approximately" are used for a given measurement, value, or characteristic, it refers to an amount within the normal operating range to obtain the desired result, but includes some variability due to inherent inaccuracies and errors within the system tolerance (such as 1% - 5%). Therefore, this description is meant only by way of example and not to limit the scope of the present invention.

Claims

1. A system for finding patterns in runtime images captured during a manufacturing process, comprising: A pattern finding tool, the pattern finding tool being trained based on one or more templates associated with one or more training images, the one or more training images containing an interesting pattern corresponding to a part in the manufacturing process; A neural network classifier, training the neural network classifier on the one or more training images; Template matching processing, wherein, during the runtime of the manufacturing process: The trained neural network classifier determines a label associated with each of the one or more templates based on the runtime image of the part, and provides a subset of the one or more templates to the pattern finding tool based on the label, and the trained pattern finding tool performs pattern matching based on the subset of the one or more templates combined with the runtime image of the part.

2. The system according to claim 1, wherein The pattern finding tool is adapted to be trained using multiple of the templates.

3. The system according to claim 1, wherein The pattern finding tool is trained on a single template.

4. The system according to claim 1, wherein The neural network includes a convolutional neural network (CNN).

5. A system for finding patterns in runtime images captured during a manufacturing process, comprising: A neural network, the neural network being trained to locate one or more candidate shapes in an image of a part in the manufacturing process, and arranged to identify the probability of the presence of the one or more candidate shapes in the runtime image during the manufacturing process, and thereby generate (a) a weighted mask having the features of the one or more candidate shapes exceeding a probability threshold, or (b) a reconstructed image, wherein the features of the model of the one or more candidate shapes are replaced in the reconstructed image, wherein the neural network identifies the presence of the features of the one or more candidate shapes exceeding the probability threshold; A pattern finding tool, training the pattern finding tool using one or more models associated with the one or more candidate shapes to find the one or more candidate shapes in (a) the weighted mask or (b) the reconstructed image; Wherein, the neural network provides data on the presence of the type of the one or more candidate shapes to the pattern finding tool, and the pattern finding tool restricts the processing to the processing related to locating the type.

6. The system according to claim 5, wherein, The neural network defines the weighted mask, wherein each pixel has a score related to the identification of the one or more candidate shapes.

7. The system according to claim 5, wherein The reconstructed image is defined as a binary image.

8. The system according to claim 5, wherein, The neural network includes a convolutional neural network (CNN).

9. A system for finding patterns in runtime images captured during a manufacturing process, comprising: A neural network, the neural network being trained to locate one or more candidate shapes in an image of a part during the manufacturing process, and being arranged to identify the probability of the presence of the one or more candidate shapes in the runtime image during the manufacturing process, and thereby generating (a) a weighted mask of the features of the one or more candidate shapes having a probability exceeding a probability threshold, and (b) a reconstructed image, wherein the features of the model of the one or more candidate shapes are replaced in the reconstructed image, wherein the neural network identifies the presence of the features of the one or more candidate shapes exceeding the probability threshold; A pattern finding tool, using one or more models associated with the one or more candidate shapes to train the pattern finding tool to find the one or more candidate shapes in (a) the weighted mask and (b) the reconstructed image; wherein the neural network provides data on the presence of the type of the one or more candidate shapes to the pattern finding tool, and the pattern finding tool restricts the processing to the processing related to locating the type.

10. A method for finding a pattern in a runtime image captured during a manufacturing process, comprising: locating, with a neural network, one or more candidate shapes in an image of a part during the manufacturing process, and identifying the probability of the presence of the one or more candidate shapes in the runtime image during the manufacturing process; and generating (a) a weighted mask of the features of the one or more candidate shapes having a probability exceeding a probability threshold, or (b) a reconstructed image, wherein the features of the model of the one or more candidate shapes are replaced in the reconstructed image, wherein the neural network identifies the presence of the features of the one or more candidate shapes exceeding the probability threshold; using a pattern finding tool to find the one or more candidate shapes in (a) the weighted mask or (b) the reconstructed image, training the pattern finding tool using one or more models relative to the one or more candidate shapes; wherein data on the presence of the type of the one or more candidate shapes is provided from the neural network to the pattern finding tool, and the processing of the pattern finding is restricted to the processing related to locating the type.

11. The method according to claim 10, further comprising defining the weighted mask with the neural network such that each pixel therein has a score related to the identification of the one or more candidate shapes.

12. The method according to claim 10, further comprising defining the reconstructed image as a binary image.

13. The method according to claim 10, wherein, The neural network includes a convolutional neural network (CNN).

14. A system for finding a pattern in a runtime image captured during a manufacturing process, comprising: A neural network, the neural network being trained to locate one or more candidate shapes in an image of a part in the manufacturing process, and arranged to identify the probability of the presence of the one or more candidate shapes in the runtime image during the manufacturing process, and thereby generate a weighted mask of the features of the one or more candidate shapes having a probability exceeding a probability threshold; A pattern finding tool, using one or more models associated with the one or more candidate shapes to train the pattern finding tool to find the one or more candidate shapes in the weighted mask; wherein the neural network provides data on the presence of the type of the one or more candidate shapes to the pattern finding tool, and the pattern finding tool restricts processing to processing related to locating the type.

15. A system for finding patterns in runtime images captured during a manufacturing process, comprising: A neural network, the neural network being trained to locate one or more candidate shapes in an image of a part in the manufacturing process, and arranged to identify the probability of the presence of the one or more candidate shapes in the runtime image during the manufacturing process, and thereby generate a reconstructed image, wherein the features of the model replacing the one or more candidate shapes in the reconstructed image, wherein the neural network identifies the presence of the features of the one or more candidate shapes exceeding a probability threshold; A pattern finding tool, using one or more models associated with the one or more candidate shapes to train the pattern finding tool to find the one or more candidate shapes in the reconstructed image; wherein the neural network provides data on the presence of the type of the one or more candidate shapes to the pattern finding tool, and the pattern finding tool restricts processing to processing related to locating the type.

16. The system according to claim 15, wherein, The reconstructed image defines a binary image.

Images