Automatic inspection and component registration

Abstract

A system and method for automated inspection and part registration is disclosed. In one embodiment, a selection of a region of a part is received within a graphical user interface that displays a representation of the part. A location of the selected region is determined based on the selection of the region of the part. Image capture parameters are determined based on the determined location. An inspection path is calculated for the part based on (a) the determined image capture parameters and (b) the location of the selected region. The calculated inspection path is executed for the part via an inspection system. [Selected Figure] Figure 1

Description

Technical Field

[0001] Cross - Reference to Related Applications This application is related to and claims the benefit of priority of U.S. Provisional Patent Application No. 62 / 617,312, filed on January 15, 2018, which is incorporated herein by reference in its entirety.

[0002] Technical Field Aspects and implementations of the present disclosure relate to, but are not limited to, data processing, and more specifically, automated inspection and component registration.

Background Art

[0003] In many industries, such as the industry related to factory - produced products, it may be convenient to inspect the produced products to ensure that there are no defects. Such inspections are often performed manually (e.g., by human inspectors), but various inaccuracies and inefficiencies may occur.

Brief Description of the Drawings

[0004] Aspects and implementations of the present disclosure will be fully understood from the detailed description provided below and from the accompanying drawings related to various aspects and implementations of the present disclosure. However, these are not intended to limit the present disclosure to specific aspects or implementations, but are merely for the purpose of facilitating explanation and understanding.

Figure 1

Figure 2

Figure 3A

Figure 3B

Figure 3C

Figure 4

Figure 5A

Figure 5B

Figure 5C

Figure 5D

Figure 5E

Figure 5F

Figure 6A

Figure 6B

Figure 7

Figure 8A

Figure 8B

Figure 9

Figure 10A

Figure 10B

Figure 10C

Figure 10D

Figure 11

Figure 12

[0005] Aspects and implementations of the present disclosure relate to automatic inspection and component registration.

[0006] The techniques described are applicable to manufactured or assembled products / objects, manufacturing processes, etc. In many scenarios, inspection of products is performed manually (e.g., by human inspectors). Such manual inspections are associated with a number of inefficiencies, such as inaccurate or ambiguous results (e.g., due to human error).

[0007] Some techniques attempt to automate certain aspects of visual inspection, but many inefficiencies still remain. For example, existing inspection techniques are generally complex. Such systems generally must be configured by highly trained / skilled users to perform properly. Thus, such systems (which must be configured by skilled / trained personnel for proper operation) may not be cost-effective for use with low-cost and / or low-profit margin components.

[0008] Accordingly, described herein are systems, methods, techniques, and related technologies for automatic inspection (e.g., component / product inspection) in various embodiments. Particularly mentioned are inspection stations or systems configured for inspection (e.g., surface inspection) of components, objects, products, etc. having complex three-dimensional structures.

[0009] As described herein, in certain embodiments, the inspection station / system referred to may incorporate hardware elements, including but not limited to, an irradiation device, sensors, articulated arms, and / or any other mechanical or robotic elements. By incorporating such technological products, a single universal inspection system can be deployed in virtually any situation / environment (e.g., with respect to the inspection of virtually any part). Thus, the described technological products can provide significant advantages and / or improvements over existing technological products in scenarios where, for example, the aspects of the production products / objects being inspected change regularly (rendering dedicated inspection systems impractical).

[0010] In addition, in certain embodiments, the described technological products can be configured in an intuitive and simple manner (e.g., even by users without specific technical training). As detailed herein, the described technological products enable a highly efficient inspection process capable of performing configurations, calibrations, etc. on virtually any part in a relatively short time. In so doing, a single inspection system / station can be utilized to inspect multiple parts / production products. Additionally, such a system can switch inspections from one production product to another with little or no "downtime" between inspections. In so doing, the described technological products can be effectively and efficiently implemented in scenarios and environments where automated inspection may be inefficient or costly.

[0011] In addition, in certain embodiments, the described technical product enables and / or facilitates the specification of various test parameters, requirements, etc., for example, in relation to the model of the part being inspected. The described technology can further automatically convert the referenced parameters, requirements, etc. into a complete inspection plan. Such an inspection plan may include specifications for operations, acquisitions, and / or tests, which are operations, processes, inspections, etc. that need to be performed (e.g., within various calculated / identified areas or regions, such as within the generated images described), and can identify the parameters for which these operations, processes, inspections, etc. are used in the test.

[0012] Therefore, it can be understood that the described technical product addresses and targets specific technical problems and long-standing drawbacks in multiple technical fields, including but not limited to manufacturing, production product inspection, and automation. As described in detail herein, the disclosed technical product provides specific technical solutions to the mentioned technical problems and insufficient needs in the mentioned technical fields, resulting in numerous advantages and improvements over conventional approaches. In addition, in various embodiments, by operating one or more of the hardware elements, components, etc. referred to herein, the described technical product is enabled, improved, and / or enhanced in the manner described herein.

[0013] An example of an inspection device (100) is shown in FIG. 1. As shown, the inspection device (100) may be provided with an optical head (110). The optical head may include or incorporate an irradiation device (112) (e.g., incandescent, infrared, laser, etc.) and / or a sensor (114) (e.g., 1D / 2D / 3D camera, etc.). Such a sensor (114) can operate in cooperation with the mentioned irradiation device (112) (which may be of different relative positions, spectra, types, etc.). In addition, the optical head (110) can be manipulated in space with respect to the object / part (130) during inspection. For example, in one embodiment, the optical head (110) can be attached to a movable arm / robot (120) that changes the position of the optical head (110) during inspection, for example.

[0014] In one embodiment, an additional / separate computing device (e.g., one or more processors as represented in FIG. 12 and described herein) can be installed or integrated into / within the optical head. Such a computing device can be configured to analyze raw data (e.g., as captured by the mentioned sensor (112)) in one embodiment. By doing so, significant efficiency can be achieved by reducing the amount of data (e.g., images, etc.) transmitted (e.g., to another system or device for additional processing, etc.). For example, by achieving significant efficiency, only parts / regions suspected of having defects (or requiring more complex processing) can be identified / transmitted for more thorough analysis (e.g., on a dedicated station / server).

[0015] In certain embodiments, the described system may further comprise or incorporate a device (150). Such a device may be a computing device or a terminal, such as those described herein (e.g., in connection with FIG. 12). Such a device (150) can be configured to visually present various information to a user (e.g., visual displays such as various models, imported images, menus of selectable options, etc.). In certain embodiments, the device (150) may also include various input / output interfaces (e.g., touch screen, keyboard, mouse, etc.) as described herein. Through such interfaces, a user / operator can provide inputs and selections, etc., as described herein, to configure, control, and / or adjust the operation of the system (100). The device (150) and / or other components of the system (100) may further include applications, programs, instructions, etc., such as component registration / inspection for configuring various components to perform one or more of the operations described herein.

[0016] The object / part (130) being inspected by the described inspection system can be positioned or oriented in several ways. For example, the part can be fixed (i.e., the inspection is performed by the optical head (110) without changing or altering the position of the object). In other embodiments, the orientation of the part can be changed / adjusted by a platform that moves or rotates, and / or a robotic arm for gripping the part and changing its orientation (e.g., in relation to the optical head or the light capture device). In yet other embodiments, the part can be placed on a moving conveyor belt (and the inspection system can inspect the part, or its various aspects, while in operation, for example). As an example, FIG. 1 shows a part (130) placed on a platform or fixture (140). In certain embodiments, the platform (140) can be configured to rotate in any number of directions and angles, etc., for example, to change the positioning of the part (130) during inspection of the part.

[0017] As described herein, in certain embodiments, by analyzing the various aspects of the mentioned part / object, accurate measurements can be reliably performed on the various aspects of the object. Additionally, the mentioned analysis may include generating the definition of various structural boundaries of the object. Such a definition of the structural boundary may be convenient, for example, to prevent or avoid damage to the inspection system or the part (such as what may occur if a collision between the operable optical head and the part takes place). Images can be captured from various directions at any number of points on the production article and / or under different conditions (such as with different types / levels of certification and / or from different directions).

[0018] In certain embodiments, it may be convenient to perform quantitatively and qualitatively different types of inspections on various regions, areas, elements, etc. of a component. For example, while performing one type of inspection on a region, area, aspect, element, etc. of a component (at a particular inspection level), it may be convenient to perform different types and levels of inspection on other regions, areas, etc. of the same component. The type of inspection associated with a particular region, area, etc. may be a function of any number of factors (e.g., size, shape, structure, material composition, etc.) in any number of combinations. For example, for a component incorporating a shiny surface and several screws, it may be convenient to use one set of inspection technology products to inspect the shiny surface (e.g., using various levels of illumination to determine if there are scratches), while using another set of inspection technology products to inspect the screws (e.g., inspecting such areas from different angles to determine if the screws are present, fully threaded, or only partially threaded, etc.).

[0019] The specific parameters / inspection technology products applied to a particular region, area, etc. of a component can be received from / based on a knowledge base (e.g., a database or storage service that may be included within and / or accessible to the device (150)) that determines various quality scores for different sets of parameters as applied to a particular inspection situation. Thus, parameters / technology products that are determined to result in high-quality / optimal results for a particular inspection situation / scenario (e.g., inspection of screws, connectors, reflective surfaces, a type of material, etc.) can be selected and incorporated into an inspection plan for the component being inspected, while inspection parameters / technology products that are determined to result in sub-optimal results (e.g., as propagated in the mentioned "knowledge base") cannot be incorporated into the inspection plan.

[0020] In certain embodiments, such areas or regions of parts may be manually identified or selected. For example, a user / administrator may be presented with a model (or other representation) of a reference part. Such a user can select or identify each area or region of the part (e.g., using a graphical user interface) such as the presence of a screw in one area, the presence of a shiny surface in another area, the presence of a connector in another area, etc. In one embodiment, when identifying / selecting such areas or regions, the user can further determine or specify various aspects of the inspection parameters applied to a particular area (e.g., for inspecting from multiple angles using various levels of illumination, for determining the presence of scratches, etc.).

[0021] In other embodiments, the mentioned knowledge base can be utilized to define the parameters applied to a particular area. For example, when selecting a particular area or region of a part that contains a screw, a query can be made to the knowledge base to identify / determine the inspection parameters applied to such an area (e.g., inspection parameters that may possibly enable the determination that a screw is present and / or fully tightened by capturing an image of the area at different angles under different levels of illumination). In so doing, with the described technical product, the inspection system can quickly register a new part to be inspected and generate an inspection plan that enables an effective and efficient inspection of that part (e.g., by inspecting each area, region, aspect, element, etc. using different inspection parameters that reflect the specifications of the particular part). Additionally, in so doing, with the described technical product, a user who may not have sufficient experience or may not have received training in preparing an inspection plan can accurately and efficiently define an executable inspection plan for the part in an effective and efficient manner. As described, the user simply needs to select or indicate various areas or regions of the part, and the described technical product can determine that inspection parameters are associated with each area, etc. (e.g., using the mentioned knowledge base).

[0022] In other embodiments, areas or regions etc. of the mentioned parts can be identified or selected in an automatic or automated manner. For example, a reference part, and / or other types or forms of the representation of the part (such as a CAD model etc.) can be received and / or provided. By analyzing or processing such mentioned parts (e.g., based on inputs received from various types of sensors), the presence of respective areas or regions etc. within the reference part (such as the presence of threads in one area, a polished surface in another area, a connector in another area etc.) can be identified / judged. In certain embodiments, then the mentioned knowledge base can be utilized to determine inspection parameters applicable to each of the identified areas or regions etc. For example, when it is determined that there are threads in a specific area or region of a part, such an area may be associated with a specific set of inspection parameters (e.g., parameters that will probably enable the determination that there are threads and / or that they are fully tightened by capturing images of the area at different angles under different levels of illumination). In other embodiments, when identifying such areas, regions, aspects, elements etc., the user can manually indicate or specify various aspects of the inspection parameters applicable to a specific area, as described herein. At that time, the inspection system can register the new part to be inspected by the described technical product and generate an effective and efficient inspection plan for the part in an automated manner.

[0023] As described herein, in certain embodiments, the described technical product can first execute a registration process for the part to be inspected. During such a registration process, the part can be introduced into the inspection system. The system analyzes, i.e., "learns", the part to generate a multi-dimensional model (MDM) that includes general / detailed descriptions of the features and aspects of the part (such as the geometric shape of the part, visual features etc.).

[0024] As described above and as shown in FIG. 1, in certain embodiments, the part to be inspected can be placed on a fixture or platform (e.g., platform (140)). If a fixture is present, it may be convenient for the inspection system of the present disclosure to analyze, i.e., “learn,” the dimensions (and other aspects) of the platform.

[0025] The mentioned fixture may be a support device or platform attached to / associated with the described inspection system. In certain embodiments, the fixture can be configured to hold the part thereon during the inspection process (e.g., in the case of a turntable). By “learning” the characteristics of the fixture, the inspection system can further take its position into account and, for example, avoid collisions with the fixture during inspection.

[0026] The 3D structure of the fixture can be determined in various ways. For example, the dimensions and position of the fixture can be identified / determined using a CAD model of the fixture, a predefined boundary 3D shape of the fixture, and / or a remote sensor (e.g., a 3D camera).

[0027] FIG. 2 is a flowchart showing a method (200) according to an example embodiment for identifying / analyzing a fixture. The method can be executed by processing logic that can include hardware (circuits, dedicated logic, etc.), software (such as that executed on a computing device as described herein), or a combination of both. In one embodiment, the method (200) is executed by one or more elements (including, but not limited to, system (100), device (150), an application executed by system (100) and / or device (150), etc.) represented and / or described in connection with FIG. 1. However, in some other embodiments, one or more blocks of FIG. 2 can be executed by another machine.

[0028] As used herein, the term "configured" encompasses its plain and ordinary meaning. In one example, a machine is configured to perform a method by virtue of having software code for the method stored in a memory accessible to the machine's processor. The processor accesses the memory and performs the method. In another example, instructions for performing a method are wired to a processor. In yet another embodiment, some of the instructions are wired and another set of instructions is stored as software code in a memory.

[0029] For simplicity of explanation, the methods are represented and described as a series of acts. However, acts in accordance with this disclosure can be performed in various orders and / or concurrently, and in combination with other acts not presented or described herein. Further, some of the acts shown may be required to implement a method in accordance with the subject matter of this disclosure. Additionally, one of ordinary skill in the art will understand and recognize that the methods can alternatively be presented as a series of interrelated states via a state diagram or events. Additionally, it should be recognized that the methods disclosed herein can be stored on a product for easy transfer and conveyance of the methods to a computing device. The term "product" as used herein is intended to encompass a computer program accessible from any computer-readable device or storage medium.

[0030] In certain embodiments, a model (e.g., a CAD model) that reflects the representation of a fixture (e.g., a fixture configured to support components within the described inspection system) can be identified. One or more dimensions of the fixture can be calculated based on the model. Alternatively, one or more inputs corresponding to one or more dimensions, shapes, templates (e.g., those that reflect the dimensions of the fixture) of the fixture can be received. One or more images of the fixture can be captured and associated with the calculated dimensions of the fixture. In some embodiments, one or more of the captured images can be presented via a graphical user interface in relation to the representation of the dimensions of the fixture (e.g., as an overlay as described herein). Based on the association of one or more of the captured images, the calculated dimensions of the fixture can be verified. For example, an input corresponding to the association of the captured image and the calculated dimensions of the fixture can be received. In certain embodiments, such an input can reflect whether the calculated dimensions are accurate / not accurate for the fixture as represented in one or more images. Additionally, in certain embodiments, the calculated dimensions can be adjusted based on the received input.

[0031] As a further example, as shown in FIG. 2, the mentioned fixture can be attached to, or introduced into, the inspection system (210). Following the introduction of the fixture, the inspection system (e.g., an optical head or a robotic arm) can be configured to move to acquire images at various positions. Thereafter, such images can be processed (e.g., by comparing the images with other data such as a CAD model) to verify that the attached fixture matches the details of the loaded fixture.

[0032] Such a verification process can be performed manually or automatically. For example, manual verification can be performed by enabling a user to input or define various aspects of the fixture by making selections such as the shape (220) and dimensions (230) of the fixture. For example, FIG. 3A shows an example of a graphical user interface (GUI) (310) that enables a user to select the shape of the fixture being referred to and define the dimensions of the fixture.

[0033] In certain embodiments, an image of the fixture in question can be captured (e.g., by the optical head (110)). Such an image can then be displayed, for example, as an overlay on top of a model of the fixture (e.g., a model as defined or selected by the user as shown in FIG. 3A, or a CAD model). FIGS. 3B and 3C illustrate GUIs ((320) and (330) respectively) showing a model of the fixture in question overlaid with an image captured by the optical head (110) of the fixture as installed within the system (100). As shown in FIGS. 3B and 3C, in certain embodiments, the image can be associated with, or enlarged by, a visual marker or “mask” (e.g., a green and / or red mask). Such a mask (e.g., a green mask) can represent the approximate location of the fixture. For example, as shown in FIG. 3B (reflecting a top view of the fixture in question), the mask (322) can reflect an area or region corresponding to the location or position of the fixture. Another such mask (324) (e.g., a red mask) can reflect an area or region where the optical head can operate safely (and thus where the fixture will not overlap). Similarly, as shown in FIG. 3C (reflecting a side view of the fixture in question), the mask (332) can reflect an area or region corresponding to the location or position of the fixture, and the mask (324) can reflect an area or region where the optical head can operate safely (and thus where the fixture will not overlap). In considering the captured / enlarged image, the user can provide an input, such as an input (240) to verify the selected / stipulated dimensions of the fixture by checking or approving a defined area (e.g., via selectable controls (326) as shown). In an automatic verification scenario, the inspection system can perform this process automatically, for example, by executing a verification plan regardless of the presence or absence of an image of the fixture and detecting the actual fixture location by comparing the resulting images.

[0034] In scenarios where the described inspection system has additional / remote sensing capabilities, such features can further improve the initial (rough) specification of the fixture. For example, the inspection system can activate such remote sensing capabilities (250) and perform improvement and fine-tuning (260) of the 3D model of the fixture while operating the optical head (within the "acceptable" space or area).

[0035] In addition, in certain embodiments, the described technical product can be configured to read CAD or other files, models, representations, etc. of the fixture (270). The dimensions reflected in such CAD / models can be used as the basis for the described verification (240).

[0036] In certain embodiments, a model (e.g., a CAD model) reflecting the representation of the mentioned part may be identified and / or one or more inputs corresponding to one or more dimensions, shapes, templates of the part (e.g., a template reflecting one or more dimensions of the part) may be received. Based on the model, one or more dimensions of the part can be calculated. An image of the part can be captured and associated with the calculated dimensions. For example, as described herein, one or more of the captured images can be presented via a graphical user interface in relation to the representation of one or more dimensions of the part. Based on the association of one or more captured images, the calculated dimensions can be verified. For example, an input corresponding to the association between one or more captured images and the calculated dimensions of the part can be received. Such an input reflects, for example, whether the calculated dimensions, shapes, etc. are accurate / not accurate for the part represented in the image. In certain embodiments, the calculated dimensions can be adjusted based on the mentioned input.

[0037] In other embodiments, inputs corresponding to the aspects of the mentioned components can be received. Such inputs may correspond to, or reflect, for example, the dimensions of the component, the shape of the component, a template reflecting one or more dimensions of the component, the selection of a CAD model of the component, etc. An image of the component can be captured and associated with one or more aspects. For example, one or more of the captured images can be presented via a graphical user interface in relation to the representation of the aspects of the component. The received aspects can be verified based on the association with the captured images. For example, user input corresponding to the association of one or more captured images with one or more aspects of the component can be received. Such user input reflects, for example, whether one or more aspects are accurate / not accurate for the component represented in the one or more images. Additionally, in some embodiments, one or more of the received aspects can be adjusted based on the user input.

[0038] Additionally, in some embodiments, one or more inputs corresponding to one or more dimensions of the component can be received. An image of the component can be captured and one or more of the images can be associated with one or more dimensions. In some embodiments, one or more of the captured images can be presented via a graphical user interface as an overlay related to the representation of the dimensions of the component. The received dimensions can be verified based on the association with one or more of the captured images. In some embodiments, user input representing at least one of the captured images as an overlay related to the representation of the dimensions of the component can be received via a graphical user interface. Additionally, in some embodiments, one or more of the received dimensions can be adjusted based on one or more user inputs as described herein.

[0039] As a further example, FIG. 4 shows an example of a process of a method (400) that enables the specification of the part to be inspected. In one embodiment, the method (400) is performed by one or more elements (including, but not limited to, the system (100), the apparatus (150), applications executed by the system (100) and / or the apparatus (150), etc.) represented and / or described in connection with FIG. 1. However, in some other embodiments, one or more blocks of FIG. 4 can be executed by another machine. It should be understood that the described part specification process is convenient not only for improving aspects of the inspection process but also for reliably avoiding collisions (e.g., for the optical head / robot arm).

[0040] FIG. 5A represents an example of a GUI (510) where a user can provide input (e.g., via an apparatus (150) configured to execute an application that provides the GUI (510)) by selecting a template (420) associated with the part to be inspected and / or by specifying dimensions (430) of the part to be inspected via the GUI mentioned. Alternatively, in some embodiments, it is possible to provide / receive input corresponding to the loading of a CAD model of the part and the use of the CAD model to specify such dimensions (440). For example, FIG. 5B represents an example of a GUI (520) where a CAD model associated with the part to be inspected can be selected to specify the shape and dimensions of the part, etc.

[0041] In the part specification process mentioned, the inspection system described can determine the exact geometric shape of the part (e.g., when initially specifying the dimensions of the part using CAD), or an approximate geometric shape of the part (e.g., in the case of templates such as box-shaped, cylindrical, semi-circular, etc.) is used. In some embodiments, various additional / remote sensors (e.g., other cameras) can be utilized to roughly estimate the dimensions of the part first (450). Such an initial estimate can be fine-tuned in the manner described above, for example, with respect to the specification of the fixture (460).

[0042] When generating a specification of a part to be inspected (which may include, for example, dimensions, shape, and / or other aspects and features of the part as described in this specification), a verification process can be executed (470). Such verification enables the generated specification to surely match the part to be inspected. Such a verification process can be executed in a similar manner as described above for fixtures, for example, by supplementing an image of a physical part to be read into an inspection system and overlaying the image on dimensions defined by the user and on a CAD model.

[0043] For example, FIGS. 5C and 5D represent examples of GUIs ((530) and (540) respectively) where a part to be inspected is verified. In such a scenario, an image of the actual part can be overlaid on the dimensions of the part, for example, as shown in FIGS. 5A or 5B and as defined in the manner described above. At that time, the user can confirm whether the specified dimensions of the part match the actual part being inspected by considering and providing various inputs. As described above, in one embodiment, the image can be enlarged by visual markings or a "mask" corresponding to the area occupied by the part (and thus not passed through by the optical head) and the area not occupied by the part (and thus through which the part can safely pass). For example, in FIG. 5C (reflecting a top view of the part to be inspected), the mask (532) reflects the area or region corresponding to the location or position of the object, while the mask (534) can reflect the area or region where the optical head can operate safely (and thus the part will not overlap). As a further example, FIG. 5D shows a mask (542) that reflects the area or region corresponding to the location or position of the part.

[0044] As yet a further example, FIGS. 5E and 5F depict examples of GUIs ((550) and (560) respectively), in which part specifications (e.g., part dimensions, shapes, and / or other aspects / features as described herein) generated based on a CAD model are verified against an image of an actual part (such as may be captured by an inspection system and / or other sensors). As shown in FIGS. 5E and 5F, an image of a part (such as may be introduced into an inspection system) can be overlaid onto a representation of the part as rendered in a CAD model. In so doing, a user can, for example, confirm that such dimensions are accurate (as rendered onto the captured image), reject that the dimensions are inaccurate (e.g., based on the captured image), and / or modify / adjust the dimensions being referred to.

[0045] When verifying the specification of a part under inspection (e.g., as reflecting part dimensions, shapes, and / or other aspects / features as described herein), the inspection system can create a dynamic plan to scan / inspect the part. Such a plan can reflect, or can include, various instructions, actions, etc. that can configure the system as described to inspect the part when implemented / executed. For example, an image can be captured from multiple directions (e.g., where there is an overlay), and / or using multiple illumination configurations (480). This image collection can be used to create a MDM (multi-dimensional model) (490). It should be understood that such a multi-dimensional model can include, and / or can incorporate, any number of part aspects, elements, etc. For example, the multi-dimensional model being referred to can introduce part aspects (and / or regions, areas, etc. thereof), including but not limited to size, shape, structure (2D, 3D, etc.), material composition, etc.

[0046] Following the registration process described, a visual representation of the part can be displayed (e.g., within a viewer / GUI). Thereby, for example, a user can identify various test requirements (e.g., at the part level).

[0047] As an example, FIG. 6A shows an example of a GUI (610), where visual representations of various dimensions / shapes of components (such as those previously input or determined) may be associated with an image of an actual component as captured at different locations, such as by overlaying the mentioned dimensions, etc. on the captured image of the component. As shown in FIG. 6A, a representation of the structure of a component (612) (reflecting, for example, dimensions input by a user) can be overlaid on an image (614) (such as an image of a component read into an inspection system and captured by an optical head at different locations / angles). In so doing, the component can be visually represented / displayed as a patch floating at approximately the location of the region in 3D space where the image is acquired, or as a collection of images to be overlaid.

[0048] In other embodiments, a component can be visually represented / displayed based on a CAD model (or other representation) of the component. For example, FIG. 6B shows an example of a GUI (620), where visual representations of various dimensions / shapes of components are shown as being generated based on a CAD model of the component. In yet other embodiments, a 3D model can be reconstructed using a preliminary scan of a component (which can be derived or calculated based on the generated MDM).

[0049] Using the GUI / viewer referred to (such as that shown in FIGS. 6A-6B), a user can provide inputs to specify various inspection / test requirements at the component level (e.g., as opposed to an image-based level). For example, by selecting or interacting with various controls (e.g., selectable buttons) such as those shown in FIGS. 6A-6B, the user can adjust (e.g., rotate, zoom, etc.) the aspect of the field of view of the component as described herein, and further adjust the component and / or the aspect of the inspection applied thereto (e.g., the method of capturing an image of the component and / or a part / element thereof, the conditions or configurations used for the aforementioned image capture and / or processing, etc.). Such specification of requirements, etc., can be performed in an automated manner (e.g., based on CAD and an initial scan) or a semi-manual manner (e.g., using the Planner Motion Assistant (PMA) technology as described below).

[0050] In certain embodiments, it may be desirable or convenient to further inspect a region of a component, e.g., to define new inspection / test requirements, in accordance with the registration process described. FIG. 7 shows an example of a process / method (700) capable of performing such further inspection. In one embodiment, the method (700) is performed by one or more of the elements represented and / or described in connection with FIG. 1 (including, but not limited to, the system (100), the apparatus (150), an application executed by the system (100) and / or the apparatus (150), etc.), although in some other embodiments, one or more blocks of FIG. 7 can be executed by a different machine.

[0051] In one embodiment, for example, the selection of a component area can be received. As described herein, such a selection can be received, for example, within a graphical user interface that represents and displays the representation of the component. Based on the received selection (e.g., the selection of a component area), the location of the selected area (and / or other aspects) can be determined. Based on the determined location, various image capture parameters can be determined, calculated, and / or identified (e.g., as described herein). For example, parameters and environments that are likely to generate a high-quality image of the selected area can be determined. In one embodiment, an inspection path (e.g., in the case of instructions regarding the manner in which an image of the selected area is acquired / captured) can be calculated for the component. As described herein, such an inspection path can be calculated, for example, based on the determined image capture parameters and / or the location of the selected area. Additionally, in one embodiment, the mentioned inspection path can be calculated to avoid collisions between elements of the inspection system (e.g., an optical head or a robotic arm) and the component being inspected. The calculated inspection path can be executed, for example, for the component introduced into the described inspection system.

[0052] As a further example, an operator of the inspection system can provide input (710) by using a mouse (or any other input device / interface) to click / select an area / region within the part viewer / GUI. For example, FIGS. 8A-8B represent examples of GUIs ((810) and (820) respectively) showing scenarios where a user has selected a particular area or region of a part. Thereby, commands can be generated / provided to operate the optical head / robot arm to obtain an image of the selected area. To complete this operation, the inspection system can elucidate / determine the location and orientation of the selected points in 3D space (720). The system can then further elucidate / solve a compatible configuration of the system's area (including, for example, parameters and environments as described herein), where a relatively good / high-quality image (taking into account various system parameters such as system range and operating distance of the optical head) can be obtained (730). It should be understood that the configurations of the system referred to may include the position of the robot and the position of the external joints. Next, the inspection system can calculate a safe (e.g., collision-free) and efficient path from the current position of the optical head to the required position (740) and execute the calculated path (750).

[0053] In one embodiment, one or more elements contained within a part (e.g., components, types and / or subtypes of components, etc.) can be identified within a model corresponding to the part in question. As described in detail herein, various test parameters can be associated with the identified elements. In one embodiment, various selections of the test parameters in question can be received (e.g., in order to associate such parameters with the identified elements). For example, an image of the part in question (e.g., as captured by the described inspection system) can be presented within a graphical user interface. Selections of various regions (e.g., selections of the image corresponding to the part in question) can be received within the graphical user interface. As described herein, such selected regions can be associated with various test parameters (e.g., test parameters corresponding to or associated with the selected elements).

[0054] As a further example, various test / inspection requirements can be defined or specified in accordance with the described registration process. FIG. 9 shows an example of a process / method (900) by which such requirements can be defined or specified. In one embodiment, the method (900) is performed by one or more of the elements represented and / or described in connection with FIG. 1 (including, but not limited to, the system (100), the apparatus (150), applications executed by the system (100) and / or the apparatus (150), etc.), but in some other embodiments, one or more blocks of FIG. 9 can be performed by another machine.

[0055] In one embodiment, an operator or administrator can input or specify various requirements and parameters of the parts. Such requirements and parameters can define or indicate various aspects of the manner in which the part (or a portion thereof) is inspected. In other embodiments, the inspection system can be configured to perform such functions in an automatic or automated manner, for example, by detecting components and / or sub-components within the part to be inspected and generating suggestions for requirements and parameters applied to the testing / inspection of those parts. In so doing, the described inspection system can generate an inspection plan that, at runtime, addresses or provides information regarding such requirements in an optimal / efficient manner.

[0056] In one embodiment, the described inspection system can analyze the mentioned MDM and / or preliminary dynamic part images to detect, segment, and / or classify common parts / elements (such as ports, screws, cables, etc.) (910)(920) (for example, using computer vision and machine (deep) learning techniques / algorithms). For example, processing the MDM generated for a part can identify ports or screws within the part that likely require inspection. In one embodiment, such elements can be detected based on the identification of equivalent elements (such as screws and ports) for other parts.

[0057] In addition, in one embodiment, the inspection system can directly extract such components (such as screws and ports that likely require inspection) by processing or analyzing the CAD model (if available). Such components may be identified or extracted based on various factors such as semantic metadata and attributes attached to the components within the CAD model (such as the model, brand, dimensions, name, etc. of the component).

[0058] The inspection system also classifies the detected components into subtypes (e.g., RJ45 port, Philips screw, etc.), and / or suggests to the user that a certain detected component should be tested (or automatically initiate this test) (930). For example, a screw can be identified and subsequently classified as a specific type of screw (e.g., Philips screw). As a further example, a port can be identified and subsequently classified as an RJ45 port. Such classification enables the application of inspection requirements associated with the component type to specific parts. Thus, in one embodiment, during the inspection of a part determined to include a specific component (e.g., Philips screws), the described inspection plan may include the determination that the screws being used are Philips screws (and not other types).

[0059] As described, in one embodiment, the user or administrator can monitor and / or provide input or feedback regarding various aspects of the described process. For example, the user can approve, edit, and / or reject various suggestions or proposals generated by the inspection system, such as suggestions regarding the application of a specific inspection to a certain component / area of a part (940).

[0060] The user can also manually define or add various inspection / test requirements (950). For example, the user can select test requirements (e.g., for a port). In doing so, the user may be further prompted to select additional options such as subtype requirements (e.g., for testing the port as an RJ45 port) (960). As an example, FIG. 10A shows an example of an interface / GUI (1010) where the user can select the type of port (here RJ45) associated with the test / inspection to be performed. As shown in FIG. 10A, the identified port (e.g., an image of the port captured by the inspection system (1012)) can be presented to the user along with various options (1014) corresponding to different port types. The user can select the port type corresponding to the port represented in the image. At that time, the test requirements / parameters associated with the selected port are applicable when performing the inspection of the mentioned component (e.g., to determine that the exact type of the port exists in the item).

[0061] In certain embodiments, the user can also specify which of the various available tests are to be performed to check for springs, housings, foreign objects, etc. (970). For example, FIG. 10B shows an example of an interface / GUI (1020) where the user can select aspects such as the manner and features to be tested / inspected. As shown in FIG. 10B, one or more tests may be selected and associated with specific components or areas within the part. It should be understood that the mentioned tests can include or reflect different types of inspection operations (e.g., image capture of the mentioned components / areas at various locations under various conditions and / or processing of the images to calculate various determinations, such as the presence of a specific component, absence of scratches, etc.).

[0062] As described, in certain embodiments, a component area or component can be selected and, for example, a certain test can be applied to the selected component / area. For example, as described herein, FIG. 10C represents an example of a GUI (1030) in which a user can select or hide an area / element (e.g., a region or component (1032)), and further, in addition to the said area, inspection tests applicable to other test requirements, environments, etc. can be defined (980).

[0063] In addition, in certain embodiments, a user can provide input / feedback such as approving or rejecting various captured images. For example, FIG. 10D represents a GUI (1040) in which various images captured by the described inspection system can be considered and approved or rejected (e.g., by the user). By way of example, various images (corresponding to images of reference elements / regions under conditions defined by the selected / judged test requirements) can be presented to the user by including and / or comprising identification within the component and / or various selected components and test parameters further selected / specified for the said element / region. The user can then select which images meet (or do not meet) the requirements of the mentioned inspection.

[0064] Based on the described determination and / or selection, an inspection plan can be generated (990). In certain embodiments, such an inspection plan can include and / or be composed of the identified / selected elements / regions of the component, various test requirements / parameters associated with the said elements / regions, and / or the selections provided (e.g., whether an image meets / does not meet the requirements of the mentioned inspection). At runtime, the information regarding the requirements is processed or provided in an optimal / efficient manner.

[0065] In one embodiment, one or more images of a component can be captured, for example, during the execution of an inspection path (as described herein). Further, in one embodiment, various previously captured images can be read. Such images can be presented via a graphical user interface. A selection can be received for at least one of the captured images. Such a selection can, for example, accept or reject one or more of the captured images. Based on the selection of at least one of the mentioned captured images, an inspection plan can be generated for the component, and such an inspection plan can further be executed for the mentioned component and / or another component. In one embodiment, such an inspection plan can be generated for the component based on the calculated inspection path and / or one or more image capture parameters, as described herein. In one embodiment, such an inspection plan can be adjusted to account for one or more limitations of the described inspection system. Additionally, in one embodiment, various test parameters can be associated with at least one of the one or more images. For example, in one embodiment, one or more first test parameters can be associated with a first image of the one or more images, and one or more second test parameters can be associated with a second image of the one or more images.

[0066] As a further example, FIG. 11 shows an example of a process (1100) capable of generating an inspection plan. In one embodiment, the method (1100) is performed by one or more elements (including, but not limited to, system (100), device (150), an application executed by system (100) and / or device (150), etc.) represented and / or described in connection with FIG. 1. However, in some other embodiments, one or more blocks of FIG. 11 can be performed by another machine.

[0067] In one embodiment, in accordance with the described test specification process (1110), the inspection system can generate a display (1120) (corresponding to an optical head configuration that may include position, illumination, etc.), from which images will be acquired during the inspection. Such a display can be presented to a user who may be provided with options to accept / reject the display (1130). The accepted display can be added to the inspection plan for the part (1140). For these images, an image-level test can be generated that meets the defined test requirements (1150) (e.g., to check for the presence or absence of a port spring in one of the images). The system can also generate a dynamic inspection plan, acquire images, and execute / perform such a plan (during which the user can accept or reject the captured images).

[0068] The inspection system can then further adjust, improve, or optimize the acquisition path generated to maximize efficiency. For example, in one embodiment, various hardware configurations (e.g., the position of the optical head, robot arm, etc.) may be desirable (e.g., for image capture under certain conditions), but may not be achievable (e.g., due to the possibility of the optical head colliding with the object being tested or other parts of the machine itself). Thus, the inspection system can achieve results that meet (or are close to) the inspection requirements while improving or optimizing such inspection plans that constitute the mentioned limitations.

[0069] In addition, in one embodiment, the inspection system can further be configured to perform as many of the mentioned inspections / tests as possible based on the captured images (e.g., instead of capturing additional images of the same / similar areas). In doing so, the operation of various components (e.g., the optical head and robot arm) can be reduced (similarly, reducing the amount of extra data acquired / processed).

[0070] Therefore, it can be recognized that the described technical product enables and / or facilitates the specification of various test parameters, requirements, etc., for example, in relation to the model of the part to be inspected. The described technology can further automatically convert the referenced parameters, requirements, etc. into a complete inspection plan. Such an inspection plan may include specifications for operations, acquisitions, and / or tests, which need to be performed (e.g., within various calculated / identified areas or regions, such as within the generated images described) for operations, processes, tests, etc., and can identify the parameters used for these operations, processes, tests, etc. in the test.

[0071] It should also be noted that although the described technical product and technology are mainly described herein with respect to production product inspection (e.g., using an inspection system as represented or described in this specification), the described technical product / technology is not so limited. Therefore, the described technical product / technology can similarly be implemented and / or utilized in other environments and situations, and for any number of additional purposes. Further technical advantages, solutions, and / or improvements (beyond those described and / or referred to in this specification) may be possible as a result of such implementation.

[0072] As an example, in one embodiment, the technology described can be used in connection with larger scale inspections such as inspections of large structures like buildings, bridges, roads, tunnels, etc. In such an embodiment, a vehicle (equipped with various cameras, sensors, etc.) can be used to operate in connection with such a structure (e.g., around and / or within it). Examples of such vehicles include, but are not limited to, manned vehicles such as automobiles and bicycles, unmanned aerial vehicles (UAVs) or “drones”, remotely controlled vehicles, unmanned vehicles such as boats, or other such operable devices. In such an embodiment, the vehicle or operable device referred to can be configured to perform an initial inspection (e.g., of a prototype or ideally constructed structure), and / or a model that defines the ideal / intended dimensions and characteristics of such a structure may be processed. In that case, an inspection plan (e.g., for the actual structure being inspected, etc.) can be calculated. As described herein, various considerations can be made when calculating such an inspection plan. For example, one or more important or critical areas of the structure being inspected can be identified, and the path along which the vehicle is operated when performing the referred inspection can be configured to prioritize such areas. As a further example, certain technical limitations of the vehicle / device may be made when calculating the referred inspection plan (e.g., battery limitations that may affect the flight time of a drone, limitations in where the vehicle can move or not move, altitude / reception limitations, etc.). In that case, the described technology enables the referred structure to be inspected in the most efficient and effective manner deemed possible. Once the referred inspection plan is calculated, the vehicle can execute the plan in a manner such as described herein (e.g., for production product inspection).

[0073] It should be understood that the components referred to herein are either integrally combined according to a particular embodiment or separable into further components. Additionally, in one embodiment, the various components of a particular device can also be executed on another machine.

[0074] One embodiment, as described herein, includes logic or a number of components, modules, or mechanisms. A module can comprise either a software module (e.g., code embedded in a machine-readable medium) or a hardware module. A “hardware module” is a tangible unit capable of performing certain operations and can be configured or arranged in a particular physical manner. In examples of various embodiments, one or more computer systems (e.g., a stand-alone computer system, a client computer system, or a server computer system) or one or more hardware modules of a computer system (e.g., a processor or a collection thereof) can be configured by software (e.g., an application or a portion of an application) operating as a hardware module to perform certain operations as described herein.

[0075] In some embodiments, a hardware module can be implemented mechanically, electronically, or any suitable combination thereof. For example, a hardware module may include dedicated circuitry or logic that is permanently configured to perform an operation. For example, a hardware module may be a special-purpose processor such as a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC). A hardware module may also include programmable logic or circuitry that is temporarily configured by software to perform a particular operation. For example, a hardware module may include software that is executed by a main processor or other programmable processor. When configured by such software, the hardware module becomes a particular machine (or a particular component of a machine) that is uniquely tuned to perform the configured function and is no longer the main processor. It will be appreciated that the decision to implement a hardware module mechanically, whether in dedicated circuitry or permanently configured circuitry, or in temporarily configured circuitry (e.g., configured by software), may be driven by cost and time considerations.

[0076] Accordingly, the phrase "hardware module" should be understood to include a tangible entity, where the entity is physically constructed, permanently configured (e.g., wired), or configured temporarily (e.g., programmed) to operate in some manner or to perform a particular operation described herein. As used herein, "wired implementation module" refers to a hardware module. Considering embodiments in which a hardware module is configured temporarily (e.g., programmed), each of the hardware modules need not be configured or instantiated in any one instance over time. For example, if a hardware module includes a main processor configured by software to be a special-purpose processor, the main processor can be configured as different special-purpose processors (e.g., including different hardware modules) at different times. Thus, software configures a particular processor to configure a particular hardware module at one time, for example, and a different hardware module at a different time.

[0077] A hardware module can provide information to and receive information from other hardware modules. Thus, the described hardware modules can be considered to be communicatively coupled. When multiple hardware modules are present simultaneously, communication can be achieved by transmitting signals (e.g., over appropriate circuits and buses) between two or more of the hardware modules. In embodiments where multiple hardware modules are configured or instantiated at different times, communication between such hardware modules can be achieved, for example, via storage and retrieval of information in a memory structure accessed by the multiple hardware modules. For example, one hardware module can perform an operation and store the output of that operation in a communicatively coupled memory device. Subsequently, a further hardware module can later access the memory device to retrieve and process the stored output. A hardware module can also initiate communication with an input / output device and operate on a resource (e.g., a collection of information).

[0078] The various operations of the example methods described herein can be performed, at least in part, by one or more processors that are temporarily (e.g., by software) or permanently configured to perform the associated operations. Whether temporarily or permanently configured, such processors can configure processor-implemented modules that operate to perform one or more of the operations or functions described herein. As used herein, "processor-implemented module" refers to a hardware module implemented using one or more processors.

[0079] Similarly, the methods described herein may be implemented, at least in part, by a processor, and a particular processor is an example of hardware. For example, at least some of the operations of the method can be performed by one or more processor implementation modules. Further, one or more processors can also operate to assist in the performance of related operations in a "cloud computing" environment or as "software as a service" (SaaS). For example, at least some of the operations can be executable by a collection of computers (e.g., machines equipped with processors), and these operations are available via a network (e.g., the Internet) and one or more appropriate interfaces (e.g., APIs).

[0080] The performance of some of the operations can be distributed among processors that exist not only within one machine but across a number of machines. In examples of some embodiments, the processor, or processor implementation module, can be located in one geographical location (e.g., a home environment, an office environment, or within a server farm). In examples of other embodiments, the processor, or processor implementation module, can be distributed across a number of geographical locations.

[0081] The modules, methods, applications, etc. described in connection with FIGS. 1-11 are implemented, in some embodiments, with respect to a machine and associated software architecture. The following sections describe representative software architectures and machine (e.g., hardware) architectures suitable for use in the disclosed embodiments.

[0082] By using a software architecture in combination with a hardware architecture, apparatuses and machines tailored for specific uses are created. For example, a specific hardware architecture coupled with a specific software architecture will result in a mobile device such as a mobile phone or a tablet device. Slightly different hardware and software architectures can lead to smart devices used in the "Internet of Things", while in another combination, a server computer for use within a cloud computing architecture is brought about. Not all combinations of such software and hardware architectures are presented herein. This is because those skilled in the art can easily understand how to implement the subject matter of the present invention in situations different from the disclosure contained herein.

[0083] FIG. 12 is a block diagram showing components of a machine (1200) according to examples of some embodiments, which can read instructions from a machine-readable medium (e.g., a machine-readable storage medium) and execute any one or more of the methods discussed herein. Specifically, FIG. 12 shows a diagrammatic representation of a machine (1200) in the form of an example of a computer system, in which instructions (1216) (e.g., software, program, application, applet, app, or other executable code) for causing the machine (1200) to perform any one or more of the methods discussed herein can be executed. The instructions (1216) transform a general-purpose machine, which is not programmed, into a particular machine programmed to perform the functions described and illustrated in the described manner. In alternative embodiments, the machine (1200) can operate as a stand-alone device or can be connected (e.g., networked) to other machines. In a networked arrangement, the machine (1200) can operate as a server machine or a client machine in a server-client network environment or as a peer machine in a peer-to-peer (or distributed) network environment. The machine (1200) can be, but is not limited to, a server computer, a client computer, a PC, a tablet computer, a laptop computer, a netbook, a set-top box (STB), a personal digital assistant (PDA), an entertainment media system, a landline telephone, a smartphone, a mobile device, a wearable device (e.g., a smartwatch), a smart home device (e.g., a smart appliance), other smart devices, a web appliance, a network router, a network switch, a network bridge, or any machine capable of continuously executing the instructions (1216) or of specifying actions taken by the machine (1200). Further, although only one machine (1200) is shown, the term "machine" is also considered to include a collection of machines (1200) that individually or in combination execute the instructions (1216) to perform one or more of the methods discussed herein.

[0084] The machine (1200) may include a processor (1210), memory / storage (1230), and I / O components (1250), which may be configured to communicate with each other via a bus (1202). In an example of one embodiment, the processor (1210) (e.g., a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an ASIC, a radio frequency integrated circuit (RFIC), other processors, or any suitable combination thereof) may include, for example, a processor (1212) and a processor (1214) capable of executing instructions (1216). The term "processor" is intended to include a multi-core processor that can include two or more independent processors (sometimes referred to as "cores") capable of executing instructions simultaneously. Although FIG. 12 shows multiple processors (1210), the machine (1200) may include a single processor with one core, a single processor with multiple cores (e.g., a multi-core processor), a multi-processor with one core, a multi-processor with multiple cores, or any combination thereof.

[0085] The memory / storage (1230) includes a memory (1232), such as main memory and other memory storage, and a storage device (1236), both of which are accessible to the processor (1210) via, for example, a bus (1202). The storage device (1236) and the memory (1232) store instructions (1216) that implement any one or more of the methods or functions described herein. The instructions (1216) may also be present, in whole or in part, within the memory (1232), within the storage device (1236), within at least one of the processors (1210) (e.g., within a cache memory of the processor), or in any suitable combination thereof during execution by the machine (1200). Accordingly, the memory (1232), the storage device (1236), and the memory of the processor (1210) are examples of machine-readable media.

[0086] As used herein, "machine-readable medium" means a device that can temporarily or permanently store instructions (e.g., instructions (1216)) and data, including, but not limited to, random access memory (RAM), read-only memory (ROM), buffer memory, flash memory, optical media, magnetic media, cache memory, other types of storage (e.g., electrically erasable programmable read-only memory (EEPROM)), and / or any suitable combination thereof. The term "machine-readable medium" should be considered to include one or more media capable of storing instructions (1216) (e.g., a centralized or distributed database, or associated cache and servers). The term "machine-readable medium" should also be considered to include any medium, or combination of media, that can store instructions (e.g., instructions (1216)) for execution by a machine (e.g., machine (1200)), such that when executed by one or more processors (e.g., processor (1210)) of the machine, the machine is caused to perform any one or more of the methods described herein. Thus, "machine-readable medium" refers to not only a single storage device or apparatus, but also a "cloud-based" storage system or a storage network that includes multiple storage devices or apparatuses. The term "machine-readable medium" excludes signals per se.

[0087] The I / O component (1250) may include various components for receiving inputs, providing outputs, generating outputs, transmitting information, exchanging information, capturing measurement values, etc. The specific I / O component (1250) included in a particular machine depends on the type of the machine. For example, a portable machine such as a mobile phone probably includes an input mechanism such as a touch input device, while a headless server machine probably does not include the touch input device. It is recognized that the I / O component (1250) may include many other components not shown in FIG. 12. The I / O component (1250) is simply classified according to a function that simplifies the following considerations, and there is no limitation on the classification method. In examples of various embodiments, the I / O component (1250) may include an output component (1252) and an input component (1254). The output component (1252) may include a visual component (e.g., a display such as a plasma display panel (PDP), a light emitting diode (LED) display, a liquid crystal display (LCD), a projector, or a cathode ray tube (CRT)), an auditory component (e.g., a speaker), a tactile component (e.g., a vibration motor, a resistance mechanism), other signal generators, etc. The input component (1254) may include an alphanumeric input component (e.g., a keyboard, a touch screen configured to receive alphanumeric inputs, an alphanumeric input component such as a photo - optical keyboard), a point - based input component (e.g., a mouse, a touch pad, a trackball, a joystick, a motion sensor, or another pointing device), a tactile input component (e.g., a physical button, a touch screen that provides the location and / or force of a touch or its gesture, or other tactile input components), an audio input component (e.g., a microphone), etc.

[0088] In an example of a further embodiment, the I / O component (1250) may include, among a variety of other components, a biometric component (1256), a motion component (1258), an environmental component (1260), or a location component (1262). For example, the biometric component (1256) may detect an expression (e.g., an expression by hand, an expression by face, an expression by voice, a gesture by body, or eye tracking), measure a biological signal (e.g., blood pressure, heart rate, body temperature, sweating, or brain waves), and may include, for example, human identification (e.g., voice identification, retina identification, face identification, fingerprint identification, or electroencephalogram-based identification). The motion component (1258) may include an acceleration sensor component (e.g., an accelerometer), a gravity sensor component, a rotation sensor component (e.g., a gyroscope), etc. As the environmental component (1260), for example, an irradiation sensor component (e.g., a photometer), a temperature sensor component (e.g., one or more thermometers for detecting ambient temperature), a humidity sensor component, a pressure sensor component (e.g., a barometer), an acoustic sensor component (e.g., one or more microphones for detecting background noise), a proximity sensor component (e.g., an infrared sensor for detecting nearby objects), a gas sensor (e.g., a gas detection sensor for detecting whether the concentration of harmful gases is safe or for measuring pollutants in the air), or other components capable of providing an index, measurement, or signal corresponding to the surrounding physical environment may be mentioned. As the location component (1262), a location sensor component (e.g., a global positioning system (GPS) receiver component), an altitude sensor component (e.g., an altimeter or a barometer for detecting atmospheric pressure from which altitude can be derived), an orientation sensor component (e.g., a magnetometer), etc. may be mentioned.

[0089] Communication can be carried out using a variety of technical products. As an I / O component (1250), there may be a communication component (1264) operable to connect a machine (1200) to a network (1280) or a device (1270) via a connection part (1282) and a connection part (1272) respectively. For example, as the communication component (1264), there may be a network interface component, or other devices suitable for matching with the network (1280). In a further example, as the communication component (1264), there may be a wired communication component, a wireless communication component, a mobile communication component, a near field communication (NFC) component, a Bluetooth (registered trademark) component (for example, Bluetooth (registered trademark) Low Energy), a Wi-Fi (registered trademark) component, and other communication components that provide communication via other modes. The device (1270) can be either another machine or various peripheral devices (for example, peripheral devices connected via USB).

[0090] Furthermore, the communication component (1264) may include a component that detects an identifier or is operable to detect an identifier. For example, the communication component (1264) may include a radio frequency identification (RFID) tag reader component, an NFC smart tag detection component, an optical reading device component (for example, an optical sensor that detects a one-dimensional barcode such as a universal product code (UPC) barcode, a two-dimensional barcode such as a quick response (QR) code, an Aztec code, a Data Matrix, a Dataglyph, a MaxiCode, a PDF417, an Ultra Code, a UCC RSS-2D barcode, and other optical codes), or an auditory detection component (for example, a microphone that identifies a tagged audio signal). In addition, various information such as location can be derived via Internet protocol (IP) geolocation, Wi-Fi signal triangulation, a specific location and a displayable NFC wireless identification signal, etc.

[0091] In examples of various embodiments, one or more portions of network (1280) may be an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless LAN (WLAN), a WAN, a wireless WAN (WWAN), a metropolitan area network (MAN), the Internet, a portion of the Internet, a portion of the public switched telephone network (PSTN), a plain old telephone service (POTS) network, a cellular telephone network, a wireless network, a Wi-Fi (registered trademark) network, other types of networks, or a combination of two or more of these networks. For example, as network (1280) or a portion thereof, a wireless network or a cellular network may be included, and the connection part (1282) may be a code division multiple access (CDMA) connection, a global system for mobile communications (GSM) connection, or other types of cellular or wireless connection parts. In this example, the connection part (1282) may implement any of various types of data transfer technologies such as single carrier radio transmission technology (1xRTT), Evolution-Data Optimized (EVDO) technology, general packet radio service (GPRS) technology, GSM enhanced data rates for GSM evolution (EDGE) technology, the 3rd Generation Partnership Project (3GPP) including 3G, 4th generation wireless (4G) networks, Universal Mobile Telecommunications System (UMTS), high speed packet access (HSPA), Worldwide Interoperability for Microwave Access (WiMAX), Long Term Evolution (LTE) standards, others defined by various standard configurations, other long distance protocols, or other data transfer technologies.

[0092] Commands (1216) can be transmitted and received over a network (1280) using a transmission medium via a network interface device (e.g., a network interface component included in a communication component (1264)) and using any one of a number of well-known transfer protocols (e.g., HTTP). Similarly, commands (1216) can be transmitted and received using a transmission medium via a connection (1272) to a device (1270) (e.g., a peer-to-peer connection). The term "transmission medium" is considered to include any non-transitory medium that can store, encode, or carry commands (1216) for execution by a machine (1200), and includes digital or analog communication signals, or other non-transitory media that facilitate the communication of software.

[0093] Throughout this specification, multiple instances may implement components, operations, or structures described as a single instance. Individual operations of one or more methods are illustrated and described as separate operations, but one or more of the individual operations can be performed concurrently, and it is not required that the operations be performed in the order shown. Structures and functions presented as separate components in example configurations can be implemented as a combination of structures or components. Similarly, structures and functions presented as a single component can be implemented as separate components. These and other variations, modifications, additions, and improvements are within the scope of the inventive subject matter herein.

[0094] Although an innovative inventive subject matter has been described with examples of particular embodiments, various modifications and changes can be made to these embodiments without departing from the broad scope of the present disclosure. Merely for convenience, the term "invention" is used, and without voluntarily limiting the scope of this application to one disclosure or innovative concept when multiple disclosures or innovative concepts are actually disclosed, such embodiments of the innovative inventive subject matter can be referred to herein individually or collectively.

[0095] The embodiments shown in this specification are described in sufficient detail to enable those skilled in the art to practice the teachings of this disclosure. Since other embodiments can be used and derived therefrom, structural and logical substitutions and changes can be made without departing from the scope of this disclosure. Therefore, the detailed description is not to be taken in a limiting sense, and the scope of the various embodiments is defined only by the claims, along with the full scope of equivalents to which the claims are entitled.

[0096] As used herein, the term "or" can be interpreted in an inclusive or exclusive sense. Further, multiple instances may be provided as one instance with respect to a resource, operation, or structure described herein. Additionally, the boundaries between various resources, operations, modules, engines, and data stores are somewhat arbitrary, and a particular operation is illustrated from the perspective of a particular exemplary configuration. Other assignments of functionality can be envisioned and can fall within the scope of various implementations of this disclosure. In general, structures and functions presented as separate resources in an exemplary configuration can be implemented as a combination of a structure or resource. Similarly, structures and functions presented as one resource can be implemented as separate resources. These and other variations, modifications, additions, and improvements are within the scope of the implementations of this disclosure as represented by the appended claims. Accordingly, this specification and the drawings are to be taken in an illustrative rather than a limiting sense.

Claims

1. A system for planning the inspection of defects in manufactured parts via an inspection system, wherein the parts include a plurality of components of different component types, and the system comprises: A processing device; A graphical user interface (GUI) representing the representation of the manufactured part; A memory coupled to the processing device and storing data representing the following: The display of the representation of the manufactured part; One or more defect types associated with the component type, wherein the one or more defect types are the types of defects suspected in the component type; At least one inspection requirement associated with the component type; An inspection technique for inspecting defect types associated with the component type; One or more image capture parameters applied to execute the inspection technique; A memory storing instructions that cause the system to perform operations when executed by the processing device; the operations include: Receiving, within the GUI, a selection of a region of the part from a user; Determining the location of the selected region based on the selection of the region of the part; Classifying one or more components within the selected region of the part that belong to a particular component type selected from among a plurality of component types, each of the plurality of component types being characterized by one or more associated defect types; Determining at least one inspection requirement for the selected region based on the particular component type, wherein the particular component type is the one or more components within the selected region classified as belonging, and the determining comprises selecting an inspection technique for the defect types associated with the component type and judging according to the one or more determined inspection requirements; Determining one or more image capture parameters to be applied to execute the inspection technique based on the determined location of the selected region and the at least one inspection requirement; Calculating an inspection path for the component based on (a) the determined image capture parameters and (b) the determined location of the selected region; and Executing the calculated inspection path on the component via the inspection system by providing one or more instructions to the inspection system to acquire one or more images of the selected region, System. **Claim 2** The memory, when executed by the processing device, further stores instructions to generate instructions for the automatic movement of the imaging device from instructions received using the graphical user interface of the system and to acquire images, and calculating the inspection path is also based on the generated instructions, the system according to claim 1. **Claim 3** Calculating the inspection path includes calculating the inspection path to avoid collisions between one or more elements of the inspection system and the component, the system according to claim 1. **Claim 4** Identifying the one or more components includes classifying subtypes of components included in the component, the system according to claim 1. **Claim 5** Identifying the one or more components includes Presenting one or more images of the component within a graphical user interface; Receiving, within the graphical user interface, a selection of one or more regions of the one or more images of the component corresponding to the one or more components; and Associating the one or more selected regions with one or more inspection requirements corresponding to the selected elements Thereby, the system according to claim 4. **Claim 6** **Claim 6** The memory further stores instructions for causing the system to perform operations, the operations comprising: capturing the one or more images of the component during the execution of the inspection path; presenting the one or more captured images via a graphical user interface; receiving a selection for at least one of the one or more captured images; and generating an inspection plan for the component based on the selection of at least one of the one or more captured images The system of claim 1. **Claim 7** The system of claim 6, wherein generating the inspection plan includes adjusting the inspection plan to account for one or more limitations of the inspection system or the possibility of high-quality image generation. **Claim 8** The memory further stores instructions for causing the system to perform operations, the operations comprising: identifying a model that reflects the representation of the component; calculating one or more dimensions of the component based on the model; capturing one or more images of the component; associating the one or more captured images with the one or more calculated dimensions; and validating the calculated dimensions based on the association of the one or more captured images The system of claim 1. **Claim 9** The memory further stores instructions for causing the system to perform operations, the operations comprising: receiving one or more inputs corresponding to one or more aspects of the component; capturing one or more images of the component using the graphical user interface; associating the one or more captured images with the one or more received aspects; and Verifying one or more received aspects based on the association of one or more captured images The system according to claim 1, comprising **Claim 10**: Receiving the one or more inputs includes receiving a selection of a CAD model of the component; Receiving one or more inputs corresponding to one or more of the dimensions or shapes of the component; or Receiving one or more inputs corresponding to a template reflecting one or more dimensions of the component; The system according to claim 9, comprising **Claim 11**: Associating the one or more captured images includes presenting, via a graphical user interface, at least one of the one or more captured images in relation to a representation of one or more aspects of the component. The system according to claim 9 **Claim 12** Presenting the presenting includes presenting an image associated with or enlarged with a visual marker or mask. The system according to claim 11 **Claim 13**: Verifying the one or more aspects includes receiving one or more user inputs corresponding to the association of one or more captured images with one or more aspects of the component, and the one or more user inputs reflecting the one or more aspects are accurate in relation to the component represented in the one or more images. The system according to claim 9 **Claim 14** The memory further stores instructions for causing the system to perform operations, the operations including Receiving one or more inputs corresponding to one or more dimensions of the component; Capturing one or more images of the component; Associating the one or more captured images with one or more dimensions; and Verifying the received dimensions based on the association of the one or more captured images. The system according to claim 1, comprising **Claim 15** The memory further stores instructions for causing the system to perform operations, the operations including: identifying a model that reflects a representation of a fixture configured to support the component within the inspection system; calculating one or more dimensions of the fixture based on the model; capturing one or more images of the fixture; associating the one or more captured images of the fixture with the calculated dimensions of the fixture; verifying the calculated dimensions of the fixture based on the association of the one or more captured images; and using the verified calculated dimensions to calculate an inspection plan. The system according to claim 1.

16. A method for planning the inspection of defects in a manufactured component, the component including a plurality of components of different component types, the method including: receiving, within a graphical user interface representing the manufactured component, a selection of a region of the component from a user; determining a location of the selected region based on the selection of the region of the component; classifying one or more components within the selected region of the component that belong to a particular component type selected from among the plurality of component types, the one or more defect types being suspected for the component type, each of the plurality of component types being characterized by one or more associated defect types; determining at least one inspection requirement for the selected region based on the particular component type, the particular component type being the one or more components within the selected region classified as belonging; the determining step including selecting an inspection technique for a defect type associated with the component type and complying with the one or more determined inspection requirements. Determining one or more image capture parameters to be applied to execute the inspection technique based on the determined location of the selected region and the at least one inspection requirement; Calculating an inspection path for the component based on (a) the determined image capture parameters and (b) the determined location of the selected region; Capturing one or more images of the component during execution of the inspection path; Presenting the one or more captured images via a graphical user interface; Receiving a selection for at least one of the one or more captured images; Associating one or more test parameters with at least one of the one or more captured images based on the selection; and Generating an inspection plan for the component based on the associated one or more test parameters A method comprising.

Images