System and method for alignment and comparison of images for reference figures and parts

Abstract

The invention provides a system and method for alignment and comparison of a reference drawing with images of a part, in particular, for alignment and comparison of a CAD drawing comprising one or more control segments with a plurality of images of a part. Each of the plurality of images corresponds to a portion of the part, and the combination of the plurality of images provides an overall representation of the part. The system comprises an imaging device for acquiring the plurality of images, a memory module for storing the plurality of images and associated imaging position data, and a processing module configured to perform an initial alignment of the CAD drawing with the plurality of images of the part, determine an initial overall alignment adjustment based on the initial alignment, and perform one or more alignment iterations until a termination condition is met. The system further comprises an output device for providing an indication of a deviation to a user based on the alignment between the CAD drawing and the part.

Description

Technical Field

[0001] The described embodiments relate to systems and methods for aligning multiple images of a computer-aided design (CAD) reference drawing with a part. Each of the multiple images may correspond to a portion of the part, and the combination of the multiple images can provide a holistic representation of the part. Background Technology

[0002] The following does not acknowledge that anything discussed below is part of the prior art or common knowledge to those skilled in the art.

[0003] A CAD drawing is a digital file that includes the 2D or 3D geometric definition of an object or part. CAD drawings can specify geometric definitions using elements called solids. For example, a CAD drawing can include solids such as lines, arcs, NURBS / splines, circles, ellipses, etc.

[0004] It is possible to check whether a part conforms to a CAD reference drawing. For example, a part can be manufactured based on the geometric definitions included in a CAD drawing. The manufactured part can then be inspected to perform quality control and ensure that the part meets functional requirements. The inspection process may include comparing the part to the geometric definitions included in the CAD drawing. Conformity may require the part to meet the tolerance requirements specified for the geometric definitions.

[0005] Traditional optical comparators use a technique called overlay or transparency, which compares a manufactured part with a scaled-down printout of its geometry. This technique relies heavily on the operator's skill in correctly aligning the parts and on the overlay / transparency. Furthermore, overlay / transparency can occur at orientations and magnification levels that are not ideally suited for the comparison process. Therefore, the drawbacks of traditional optical comparators include low accuracy and operator dependence on results. Summary of the Invention

[0006] The following introduction is provided to guide the reader toward a more detailed discussion that follows. This introduction is not intended to limit or define any claimed or unclaimed invention. One or more inventions may exist in any combination or sub-combination of the elements or process steps disclosed in any part of this document (including its claims and drawings).

[0007] According to a broad aspect, a system is provided for aligning and comparing a CAD drawing including one or more control segments with multiple images of a part, wherein each of the multiple images corresponds to a portion of the part, and the combination of the multiple images provides a holistic representation of the part. The system includes an imaging device for acquiring the multiple images, wherein each of the multiple images corresponds to at least one of the one or more control segments. The system also includes a memory module for storing the multiple images and imaging position data associated with the multiple images, wherein the imaging position data for each image indicates the relative position of the part with respect to the imaging device during imaging. Furthermore, the system includes a processing module configured to perform an initial alignment of the CAD drawing with the multiple images of the part, determine an initial overall alignment adjustment based on the initial alignment, and perform one or more alignment iterations until a termination condition is met. Each alignment iteration includes: for each of the multiple images of the part, determining a portion of the CAD drawing corresponding to a visible portion of the part in that image of the part, and for each control segment included in a portion of the CAD drawing: if an alignment adjustment corresponding to the control segment has not been determined during the alignment iteration, locating a measurement point on the image of the part corresponding to the control segment; and determining an alignment adjustment between the measurement point and the control segment. Each alignment iteration includes: changing the total alignment adjustment based on the alignment adjustment determined for that alignment iteration; and determining whether a termination condition is met. The system also includes an output device for providing the user with an indication of deviation based on the alignment between the CAD drawing and multiple images.

[0008] In one or more embodiments, one or more control segments include a single point at multiple locations.

[0009] In one or more embodiments, one or more control segments include a plurality of points located along a CAD entity, wherein the spacing between the plurality of points is based on the resolution of a plurality of images.

[0010] In one or more embodiments, the initial alignment is based on user input received at the input device.

[0011] In one or more embodiments, initial alignment is performed automatically using pattern matching.

[0012] In one or more embodiments, the measurement points are located using pattern matching.

[0013] In one or more embodiments, the measurement point is located using a search range corresponding to the control segment.

[0014] In one or more embodiments, if an alignment adjustment corresponding to the control segment has been determined during the alignment iteration and a skip parameter has been set, the processing module is also configured to perform a weighted average of multiple alignment adjustments corresponding to the control segment.

[0015] In one or more embodiments, the alignment adjustment between the measurement point and the control segment is determined based on a deviation nominal method.

[0016] In one or more embodiments, the alignment adjustment between the measurement point and the control segment is determined based on a fitting tolerance method.

[0017] In one or more embodiments, total alignment adjustment includes translation adjustment and / or rotation adjustment.

[0018] In one or more embodiments, the alignment iteration termination condition includes a maximum iteration threshold and a minimum total alignment adjustment threshold.

[0019] In one or more embodiments, providing the user with an indication of deviation based on alignment includes verifying for each control segment whether the part is within the tolerance zone associated with that control segment.

[0020] According to a broad aspect, a method is provided for aligning and comparing a CAD drawing including one or more control segments with multiple images of a part, wherein each of the multiple images corresponds to a portion of the part, and the combination of the multiple images provides a holistic representation of the part. The method includes: receiving multiple images, wherein each of the multiple images corresponds to at least one of the one or more control segments; and storing the multiple images and imaging position data associated with the multiple images, wherein the imaging position data for each image indicates the relative position of the part with respect to an imaging device during imaging. The method further includes: performing an initial alignment of the CAD drawing with the multiple images of the part; determining an initial overall alignment adjustment based on the initial alignment; and performing one or more alignment iterations until a termination condition is met. Each alignment iteration includes: for each of the multiple images of the part, determining a portion of the CAD drawing corresponding to a visible portion of the part in that image of the part, and for each control segment included in the portion of the CAD drawing: if an alignment adjustment corresponding to the control segment has not been determined during the alignment iteration, locating a measurement point on the image of the part corresponding to the control segment; and determining an alignment adjustment between the measurement point and the control segment. Each alignment iteration further includes: changing the overall alignment adjustment based on the alignment adjustment determined for that alignment iteration; and determining whether a termination condition is met. The method also includes providing the user with an indication of deviation on the output device, wherein the indication is based on the alignment between the CAD drawing and multiple images.

[0021] In one or more embodiments, one or more control segments include a single point at multiple locations.

[0022] In one or more embodiments, one or more control segments include a plurality of points located along a CAD entity, wherein the spacing between the plurality of points is based on the resolution of a plurality of images.

[0023] In one or more embodiments, the initial alignment is based on user input received at the input device.

[0024] In one or more embodiments, performing initial alignment includes automatically performing initial alignment using pattern matching.

[0025] In one or more embodiments, locating measurement points includes using pattern matching.

[0026] In one or more embodiments, locating the measurement point includes using a search range corresponding to the control segment.

[0027] In one or more embodiments, if an alignment adjustment corresponding to the control segment has been determined during the alignment iteration and a skip parameter has been set, the method further includes performing a weighted average of multiple alignment adjustments corresponding to the control segment.

[0028] In one or more embodiments, determining the alignment adjustment between the measurement point and the control segment includes determining the alignment adjustment based on a deviation nominal method.

[0029] In one or more embodiments, determining the alignment adjustment between the measurement point and the control segment includes determining the alignment adjustment based on a fitting tolerance method.

[0030] In one or more embodiments, total alignment adjustment includes translation adjustment and / or rotation adjustment.

[0031] In one or more embodiments, the alignment iteration termination condition includes a maximum iteration threshold and a minimum total alignment adjustment threshold.

[0032] In one or more embodiments, providing the user with an indication of deviation includes verifying for each control segment whether the part is within the tolerance zone associated with that control segment.

[0033] Those skilled in the art will understand that the systems or methods disclosed herein may embody any one or more features contained herein, and that these features may be used in any particular combination or sub-combination.

[0034] These and other aspects and features of the various embodiments will be described in more detail below. Attached Figure Description

[0035] The accompanying drawings are used to illustrate various examples of the systems and methods taught in this specification and are not intended to limit the scope of the teachings in any way.

[0036] Figure 1 A block diagram of a system for acquiring multiple images of a part and performing alignment of a CAD drawing with the multiple images, according to an example embodiment, is shown.

[0037] Figure 2 A block diagram of a control unit according to an example embodiment is shown.

[0038] Figure 3 An example of a corrected and enhanced image of a part is shown.

[0039] Figure 4 A schematic representation of an example control segment is shown.

[0040] Figure 5A It shows the superimposed on Figure 3 An example of a CAD drawing on a corrected and enhanced image of the part shown.

[0041] Figure 5B The aligned Figure 5A Examples of CAD drawings.

[0042] Figure 6 A schematic representation of an example of a CAD drawing aligned with an image of a part is shown.

[0043] Figure 7 A schematic representation of an example of a CAD drawing aligned with an image of a part is shown.

[0044] Figure 8 A schematic representation of an example of a CAD drawing aligned with an image of a part is shown.

[0045] Figure 9A An example of a portion of a CAD drawing superimposed on a corrected and enhanced image of a part is shown.

[0046] Figure 9B It shows the superimposed on Figure 9A Another portion of the part shown is on a corrected and enhanced image. Figure 9A An example of another part of the CAD drawing shown.

[0047] Figure 10A An example image of the part is shown at 10x magnification.

[0048] Figure 10B It shows Figure 10A Example image of the part shown at 20x magnification.

[0049] Figure 10C It shows Figure 10A and Figure 10B Example image of the part shown at 50x magnification.

[0050] Figure 11 This is a flowchart illustrating an example method for aligning a CAD drawing with multiple images of a part according to the present disclosure.

[0051] Figure 12 This is a flowchart illustrating an example method for performing alignment iteration according to the present disclosure.

[0052] Figure 13 An example of a CAD drawing superimposed on an image of a part is shown.

[0053] Figure 14 A schematic representation of an example of a CAD drawing aligned with an image of a part is shown.

[0054] Figure 15 A schematic representation of an example of a CAD drawing aligned with an image of a part is shown. Detailed Implementation

[0055] The accompanying drawings described below are for illustrative purposes and not for limiting the various examples of the embodiments described herein. For the sake of brevity and clarity, the elements shown in the drawings are not necessarily drawn to scale. For clarity, the dimensions of some elements may be exaggerated relative to others. It will be understood that, for the sake of brevity and clarity, reference numerals may be repeated between drawings where deemed appropriate to indicate corresponding or similar elements or steps.

[0056] Furthermore, numerous specific details are set forth in order to provide a thorough understanding of the embodiments described herein. However, those skilled in the art will understand that the embodiments described herein can be practiced without these specific details. In other instances, well-known methods, processes, and components have not been described in detail so as not to obscure the embodiments described herein. Moreover, this description should not be construed as limiting the scope of the embodiments described herein.

[0057] The following describes examples of various systems or methods to provide embodiments of the claimed subject matter. No embodiment described below limits any claimed subject matter, and any claimed subject matter may cover methods or systems other than those described below. The claimed subject matter is not limited to systems or methods having all the features of any of the systems or methods described below, nor is it limited to the common features of the multiple or all the devices or methods described below. The systems or methods described below may not be embodiments cited in any claimed subject matter. Any subject matter disclosed in the systems or methods described below that is not claimed in this document may be the subject matter of another protective document, such as a successor patent application, and the applicant, inventor, or owner does not intend to waive, deny, or publicly offer any such subject matter by disclosing it in this document.

[0058] Unless otherwise expressly stated, the terms “an embodiment,” “an embodiment,” “various embodiments,” “the embodiment,” “the respective embodiments,” “one or more embodiments,” “some embodiments,” and “an embodiment” refer to “one or more (but not all) embodiments of the present invention.”

[0059] Furthermore, as used herein, the term "and / or" is intended to indicate inclusion or. That is, for example, "X and / or Y" is intended to mean X or Y or both. As another example, "X, Y and / or Z" is intended to mean X or Y or Z or any combination thereof.

[0060] Unless otherwise expressly stated, the terms “including,” “comprising,” and variations thereof mean “including, but not limited to.” Unless otherwise expressly stated, the list of items does not imply that any or all items are mutually exclusive. Unless otherwise expressly stated, the terms “a,” “an,” and “the” mean “one or more.”

[0061] It should be noted that degree terms such as “basically,” “about,” and “approximately” used in this article refer to a reasonable amount of deviation from the modified term so that the final result does not change significantly. These degree terms can also be interpreted as including the deviation from the modified term, provided that such deviation does not negate the meaning of the modified term.

[0062] It should also be noted that the terms “coupled” or “coupled” as used herein can have several different meanings depending on the context in which they are used. For example, the terms “coupled” or “coupled” can be used to indicate that an element or device can electrically, optically, or wirelessly transmit data to and receive data from another element or device.

[0063] Furthermore, any description of numerical ranges by endpoints in this document includes all numbers and fractions contained within that range (e.g., 1 to 5 includes 1, 1.5, 2, 2.75, 3, 3.90, 4, and 5). It should also be understood that all numbers and their fractions are presumed to be modified by the term “about,” meaning that at most a certain amount of change may be made to the numbers mentioned if the final result does not change significantly.

[0064] Example embodiments of the systems and methods described herein can be implemented as a combination of hardware or software. In some cases, the example embodiments described herein can be implemented at least in part using one or more computer programs that execute on one or more programmable devices including at least one processing element and a data storage element (including volatile memory, non-volatile memory, storage elements, or any combination thereof). Depending on the nature of the device, these devices may also have at least one input device (e.g., a button keyboard, mouse, touchscreen, etc.) and at least one output device (e.g., a display screen, printer, radio, etc.).

[0065] It should also be noted that there may be elements for implementing at least a portion of one of the embodiments described herein, which can be implemented via software written in a high-level computer programming language (e.g., object-oriented programming). Therefore, the program code can be written in C, C++, or any other suitable programming language, and may include modules or classes, as known to those skilled in object-oriented programming. Alternatively or otherwise, some of these elements implemented via software may be written in assembly language, machine language, or firmware as needed. In any case, the language can be a compiled or interpreted language.

[0066] At least some of these software programs may be stored on a storage medium (e.g., a computer-readable medium, such as, but not limited to, ROM, disk, optical disc) or on a device readable by a general-purpose or special-purpose programmable device. When read by a programmable device, the software program code configures the programmable device to operate in a new, specific, and predefined manner to perform at least one of the methods described herein.

[0067] Furthermore, at least some programs associated with the systems and methods of the embodiments described herein can be distributed to a computer program product including a computer-readable medium carrying computer-usable instructions for one or more processors. The medium can be provided in various forms, including non-transitory forms such as, but not limited to, one or more floppy disks, optical disks, magnetic tapes, chips, and magnetic and electronic storage devices.

[0068] The embodiments described herein relate to systems and methods for aligning and comparing one or more images of a CAD drawing and a part. If the part is small enough to fit within the optical field of view of the imaging device used, a single image of the part can include all relevant features for checking the conformity of the part with its CAD drawing. The CAD drawing of the part can be overlaid on a single image and aligned with the image. Based on the alignment, the conformity of the part with its CAD drawing at one or more locations can be determined.

[0069] However, if a part is too large to fit within the optical field of view of the imaging device used, multiple images of the part may be needed to capture all relevant features. In some examples, a part may fit within the optical field of view at a low optical magnification level. However, for parts requiring high-precision manufacturing, a high optical magnification level may be necessary to check for compliance with stringent tolerance constraints. Higher optical magnification levels can result in a reduced optical field of view, thus requiring multiple images to capture all relevant features of the part. In some cases, the height or depth of the part may exceed the depth of field of the imaging device. Multiple images along the Z-axis or depth axis may be needed to capture all relevant features of the part.

[0070] Each of multiple images can correspond to a portion of a part, and the combination of multiple images can provide a complete representation of the part. CAD drawings can be aligned using a single image, and that alignment can then be used to perform conformity measurements on multiple images. However, in this case, the alignment between the CAD drawing and the part may deteriorate for one or more of the multiple images. This can lead to erroneous results when determining conformity, as referenced below. Figure 9B Further detailed description.

[0071] The described embodiments enable the alignment of CAD drawings with multiple images of a part. After alignment, the described embodiments can automatically determine the conformity of the part to its CAD data at one or more specific locations or over the entire geometric definition of the part, within user-specified tolerances. Therefore, the described embodiments can achieve alignment (and subsequent conformity determination) for situations where multiple images of a part are required (e.g., due to field-of-view or depth-of-field limitations of the imaging device).

[0072] The described embodiments can be implemented during the manufacturing process, wherein each part or a portion of a part can be inspected as they move along the production line. The described embodiments can be operatively connected to a control system to reject parts that are outside tolerance values. Therefore, the described embodiments can increase manufacturing throughput because there is no need to remove parts from the production line for inspection.

[0073] For reference Figure 1 The diagram illustrates a block diagram 100 of a system for acquiring multiple images of a part and performing alignment of a CAD drawing with the multiple images, according to an example embodiment. Block diagram 100 includes a control unit 105, an output device 110, input devices 115a and 115b, a part 120, an imaging device 125, an illumination source 130, platforms 135a-135c, and an encoder 140.

[0074] The illumination source 130, component 120, and imaging device 125 are mounted using platforms 135a, 135b, and 135c, respectively. In some embodiments, a single combined platform may be used to mount the illumination source 130, component 120, and imaging device 125. One or more of platforms 135a-135c may be capable of controlled movement to provide relative motion between component 120 and imaging device 125. One or more of platforms 135a-135c may include motor assemblies to provide controlled movement in one or more of the x, y, and z directions. Control unit 105 may provide control commands to control the movement of the platforms. In some embodiments, the controlled movement of one or more of platforms 135a-135c may be manual.

[0075] One or more encoders may be used to measure the movement of platforms 135a-135c. For example, encoder 140 may provide the control unit 105 with imaging position data indicating the relative position of part 120 with respect to imaging device 125 during imaging.

[0076] The illumination source 130 can illuminate the part 120 while the imaging device 125 captures multiple images of the part 120. In some embodiments, the control unit 105 can provide control commands to control the illumination source 130 to provide uniform and reproducible illumination to the part 120 during imaging.

[0077] Imaging device 125 can capture a high-resolution digital image stream of part 120. The high-resolution digital images can be captured at different relative x, y, z positions of part 120 relative to imaging device 125. In some embodiments, imaging device 125 can capture high-resolution still images.

[0078] Output device 110 may include a display, such as an LED- or LCD-based display. In some embodiments, output device 110 may include a gesture-enabled touch-sensitive display. Output device 110 may provide a display of one or more of a plurality of images, an overlay of a CAD drawing, a portion of a CAD drawing, or a portion of a CAD drawing with one or more of the plurality of images, and an indication of the conformity of a part with its CAD data. In some embodiments, output device 110 may provide a real-time display of video of part 120 captured by imaging device 125 at the full frame rate of the captured video. Output device 110 may provide a display of multiple images at a full 1:1 resolution, i.e., each pixel in the image is displayed using one pixel of output device 110. In some embodiments, output device 110 may be integrated into control unit 105.

[0079] Input devices 115a and 115b may include one or more of a mouse, keyboard, touchscreen, scroll wheel, touchpad, and trackball. In some embodiments, some of these components may be integrated with each other. A user may use one or more input devices 115a-115b to provide various user inputs to control unit 105. In some embodiments, input devices 115a and 115b may be integrated into control unit 105.

[0080] For reference Figure 2 A block diagram of a control unit 105 according to an example embodiment is shown. The control unit 105 may include a personal computer, workstation, server, portable computer, or a combination thereof. In some embodiments, the control unit 105 may be physically located remotely from any one or more of the output device 110, input devices 115a and 115b, imaging device 125, illumination source 130, and platforms 135a-135c. In the example shown, the control unit 105 includes a communication unit 204, a power unit 206, a processor unit 208, a memory unit 210, an I / O unit 212, and a user interface engine 214.

[0081] Processor unit 208 controls the operation of control unit 105. Processor unit 208 can be any suitable processor, controller, or digital signal processor capable of providing sufficient processing power according to the configuration, purpose, and requirements of control unit 105 known to those skilled in the art. For example, processor unit 208 can be a high-performance general-purpose processor. In alternative embodiments, processor unit 208 may include more than one processor, wherein each processor is configured to perform different dedicated tasks. In alternative embodiments, dedicated hardware may be used to provide some of the functions provided by processor unit 208. For example, processor unit 208 may include a standard processor, such as... processor or processor.

[0082] Communication unit 204 may include wired or wireless connectivity. Communication unit 204 may be used by control unit 105 to communicate with other devices or computers. For example, control unit 105 may use communication unit 204 to communicate with one or more of output devices 110, input devices 115a and 115b, imaging device 125, illumination source 130, platform 135a-135c, and encoder 140. Communication may be via a network, such as any network or network component capable of carrying data, including the Internet, Ethernet, fiber optic, mobile, wireless (e.g., Wi-Fi, WiMAX), local area network (LAN), wide area network (WAN), direct point-to-point connection, mobile data network (e.g., Universal Mobile Telecommunications System (UMTS), 3GPP Advanced Long Term Evolution (LTE Advanced), Global Microwave Access Interoperability (WiMAX), etc.) and others (including any combination thereof).

[0083] Processor unit 208 may also execute a user interface engine 214 for generating various user interfaces. User interface engine 214 may be configured to provide a user interface for displaying various menu options to a user, such as defining control segments for a CAD drawing or defining tolerances associated with control segments. In some embodiments, user interface engine 214 may be configured to enable a user to manually align an image of a CAD drawing and a part, perform on-screen measurements based on multiple images of the part, and determine the deviation of the part from its CAD drawing.

[0084] The power unit 206 can be any suitable power source that supplies power to the control unit 105, such as a power adapter or a rechargeable battery pack, depending on the implementation of the control unit 105, as is known to those skilled in the art.

[0085] I / O unit 212 may include an interface to couple one or more output devices 110 and input devices 115a, 115b. I / O unit 212 enables processor unit 208 to receive user input from input devices 115a, 115b. I / O unit 212 also enables processor unit 208 to provide output display to the user on output device 110.

[0086] The memory unit 210 includes software code for implementing the operating system 220, program 222, database 224, image processing engine 226, control segment engine 228, comparison engine 230, and measurement engine 232.

[0087] Memory unit 210 may include RAM, ROM, one or more hard disk drives, one or more flash drives, or other suitable data storage elements, such as disk drives. Memory unit 210 is used to store operating system 220 and programs 222, which are generally known to those skilled in the art. For example, operating system 220 provides various basic operating procedures for control unit 105. For example, operating system 220 may be such as... Server operating system or Red An operating system such as Enterprise Linux (RHEL), or another operating system.

[0088] Program 222 includes various programs to enable control unit 105 to perform various functions, such as, but not limited to, receiving multiple images of a CAD drawing and a part, providing a display of the CAD drawing and the multiple images, performing alignment of the CAD drawing with the multiple images of the part, and providing an indication of the deviation of the part from its CAD drawing. Program 222 may also include an optical calibration program to determine calibration for the distance between a pair of pixels in the received image and the physical distance between corresponding points on the part. In some embodiments, program 222 may include an edge detection program to detect edges of the part in the image. Program 222 may also include a pattern matching program to perform image analysis to find matching points on the image.

[0089] Database 224 can store data, including received CAD drawings and multiple images, imaging position data associated with the multiple images, any measurements of parts based on the received images, and any indications of part deviations based on the alignment of the CAD drawing with the multiple images. Database 224 can store data such as... Figure 2 It can run on the control unit 105 shown, or it can run independently on a database server that communicates with the control unit 105 via a network.

[0090] Image processing engine 226 can perform real-time image correction operations on the image stream received from imaging device 125. In some embodiments, the received images may be stored in database 224, and image processing engine 226 can perform image correction operations on images retrieved from database 224. Distortions in the image may be caused by optical lenses used in imaging device 125 or by defective pixels in optical sensors used in imaging device 125. Image correction operations can be performed to obtain a geometrically correct image of the part, i.e., an image in which all pixels correspond to the same real-world size. Example image correction operations are described in further detail in U.S. Patent No. 8,917,320.

[0091] Furthermore, image processing engine 226 can perform image enhancement on the corrected image. Image processing engine 226 can perform real-time image enhancement operations on image streams received from imaging device 125 or images retrieved from database 224. Image enhancement operations may include one or more of noise reduction, uneven illumination correction, edge sharpening, and color correction. Example image enhancement operations are described in further detail in U.S. Patent No. 8,917,320.

[0092] Control unit 105 can use corrected and enhanced images to perform alignment between CAD drawings and images. For example, Figure 3 An example corrected and enhanced image 300 of a patellofemoral joint implant is shown. In this example, the patellofemoral joint implant part is adapted to the optical field of view of the imaging device 125, and a single image captures the entire part.

[0093] Control unit 105 may use one or more control segments associated with a CAD drawing to perform alignment of the CAD drawing with multiple images of a part. The control segments may be discrete points on the CAD data and may have associated orientations and tolerances. In some embodiments, the control segment may be perpendicular to the CAD geometry at that location. The tolerance associated with the control segment may be a two-way tolerance (e.g., +0.005″ / -0.001″). Two-way tolerances may be positive / positive tolerances (e.g., +0.005″ / +0.001″), negative / negative tolerances (e.g., -0.001″ / -0.005″), or zero-value tolerances (e.g., +0.005″ / 0″ or 0″ / -0.005″). One or more control segments may be located on the CAD data. For example, the CAD drawing may include multiple control segments located at individual points along the CAD drawing, these control segments including features that are important for aligning the CAD drawing with an image of a part or for measuring deviations of the part from its CAD drawing. In some embodiments, the control segment may include a plurality of points along the CAD entity, spaced apart according to the resolution of the image received from the imaging device 125. In some embodiments, the control segment may be located at a distance from the CAD entity and may not be located on the CAD entity itself.

[0094] Each of the multiple control segments can include different tolerances associated with it. Control segments can be user-defined or can be included in CAD drawings used to manufacture the parts.

[0095] In some embodiments, the control segment engine 228 enables users to define one or more control segments for a CAD drawing. Example methods for setting up control segments are described in further detail in U.S. Patent No. 8,780,223. In some embodiments, the control segment engine 228 may use, for example, an adapted automatic discretization scheme to automatically set up control segments.

[0096] For reference Figure 4 The diagram illustrates a schematic representation of an example control segment 400. Control segment 400 is located at position 410 on CAD data 405. The control segment includes a positive direction tolerance 415 and a negative direction tolerance 420.

[0097] Now refer to again Figure 2 The comparison engine 230 can perform alignment and comparison of CAD drawings, including one or more control segments, with multiple images of the part. The CAD drawing may include the geometric definition of the imaged part in real-world units, such as inches or millimeters. The CAD drawing may be included in a CAD file stored in database 224, for example, in DXF or DWG CAD file format. The control unit 105 can use an optical calibration program included in program 222 to read in the CAD file of the imaged part and determine the pixel representation of the CAD drawing. In some embodiments, the control unit 105 can use the optical calibration program to automatically scale the content of the CAD drawing and provide a display of the scaled CAD drawing overlaid on an image of the part or a live image stream.

[0098] Comparison Engine 230 can perform alignment between CAD drawings and images of parts. For example, Figure 5A It shows the superimposed on Figure 3 Example 500 of CAD drawing 505 on a corrected and enhanced image of the patellofemoral joint implant. Alignment can be performed using a manual or automatic alignment process.

[0099] During manual alignment, a CAD drawing showing an image overlaid on the part can be displayed to the user on output device 110. The user can provide alignment commands (e.g., xy translation commands or rotation commands) using input devices 115a and 115b.

[0100] During automatic alignment, comparison engine 230 can use the pattern matching program of program 222 to locate matching pattern points between the CAD drawing and the image of the part, and perform alignment based on the located matching pattern points. In some embodiments, a manual alignment process can be used when the CAD drawing and the image of the part (e.g., a first image) are first aligned. The result of the manual alignment (e.g., the position and orientation of the CAD drawing relative to the first image) can be stored in database 224. For subsequent comparison of the CAD drawing with any new image of the part, comparison engine 230 can use the pattern matching program to determine translation and / or rotation differences between the first and new images of the part. Comparison engine 230 can then perform automatic alignment by applying the translation and / or rotation differences to the stored position and orientation of the CAD drawing. In some embodiments, comparison engine 230 can perform the automatic alignment process without providing a display on output device 110.

[0101] Now back Figure 2 The measurement engine 232 can perform measurements after the CAD drawing and the image of the part are aligned. (See reference below.) Figure 6 The diagram 600 shows an example of a CAD drawing aligned with an image of part 605. The CAD drawing may include a control segment 610 at a nominal location 615. The measurement engine 232 may determine measurement points on the edge of part 605 corresponding to the nominal location 615 of the control segment 610. The edge of part 605 may be automatically determined using an edge detection program included in program 222.

[0102] In some embodiments, the measurement engine 232 may use a search range corresponding to a control segment to determine measurement points. The search range may be a line that passes through the nominal location of the control segment and is perpendicular to the CAD entity at that nominal location. Measurement points may be determined as the intersection of the search range and an edge of the part. For example, Figure 6 The search range 620 corresponding to control segment 610 is shown. The search range 620 passes through nominal position 615 and is perpendicular to CAD entity 625 at nominal position 615. Measurement engine 232 can determine measurement point 645 based on the intersection of search range 620 and edge 640 of part 605.

[0103] Measurement engine 232 can determine measurement points along the length of the search range. The length of the search range can be based on the tolerance associated with the control segment. For example, measurement engine 232 can set the length of search range 620 to... Figure 6 The distance between the positive tolerance 630 and the negative tolerance 635 shown is five times. In some embodiments, the measurement engine 232 may set the length of the search range 620 based on user input received at input device 115a or 115b.

[0104] The search range 620 can also have associated positive and negative directions. For Figure 6 In the example shown, the positive direction of the search range 620 is along the material away from part 605. For an example where the search range intersects multiple edges of the part, the associated direction can be used to determine which of the multiple edges to locate the measurement point on.

[0105] For reference Figure 7The diagram 700 shows an example of a CAD drawing aligned with an image of part 705. The CAD drawing may include a control segment 710 at a nominal location 715. Measurement engine 232 can determine a measurement point 745 on part 705 corresponding to the nominal location 715 of the control segment 710. Measurement engine 232 can determine measurement point 745 using a search range 720 corresponding to the control segment 710. Search range 720 passes through nominal location 715 and is perpendicular to CAD entity 725 at nominal location 715. Search range 720 intersects edge 740 of part 705 at point 745 and edge 750 at point 755. In this example, point 745 corresponds to the search range direction away from part 705 (a transition from black to white). Conversely, point 755 corresponds to the search range direction towards part 705 (a transition from white to black). Therefore, between points 745 and 755, measurement engine 232 can determine point 745 as the measurement point.

[0106] In some embodiments, the measurement engine 232 may use additional criteria to determine measurement points using a search range. For example, the measurement engine 232 may use an edge threshold to determine whether a point along the search range is an edge point that must be considered when determining the measurement point. For example, the edge threshold may specify a specific change in the black / white contrast level at the point to be considered an edge point. In some examples, multiple points may be located using a search range (including a search range direction criterion), and the measurement engine 232 may determine the measurement point as the point closest to the nominal position of the corresponding control segment.

[0107] In some embodiments, the measurement engine 232 may use the pattern matching program of program 222 to determine measurement points. For example, the measurement engine 232 may define a search area centered on nominal location 615. Furthermore, the measurement engine 232 may use the pattern matching program to match the pattern of part 605 in the search area with the pattern of the CAD drawing corresponding to control segment 610. The measurement engine may determine measurement points based on the best matching pattern located using the pattern matching program.

[0108] Based on the determined measurement points, the measurement engine 232 can determine the deviation of the part from its CAD drawing by measuring the distance between the measurement points and the nominal positions corresponding to the control positions. (See also...) Figure 8The diagram illustrates a schematic representation 800 of an example CAD drawing aligned with an image of part 805. The CAD drawing may include a control segment 810 at a nominal location 815 on CAD entity 825. Measurement engine 232 can determine a measurement point 845 corresponding to the intersection of control segment 810 and edge 840 of part 805. Measurement engine 232 can determine measurement point 845 using a search range associated with the control segment, as described above. Measurement engine 232 can also determine the deviation 860 of part 805 from its CAD drawing at control segment 810. Measurement engine 232 can determine the corresponding physical deviation of the part from its geometric definition based on a calibration relationship between physical distance and the distance between a pair of pixels in an optical image.

[0109] In some embodiments, the measuring engine 232 may indicate a positive deviation for deviations measured along the control segment in a direction away from the material of the part, and a negative deviation for deviations measured in the opposite direction. Figure 8 In the example, measurement engine 232 can indicate deviation 860 as a negative deviation.

[0110] In some embodiments, the CAD drawing may include multiple control segments, and the measurement engine 232 may determine the deviation of each control segment. For example, as Figure 5A and Figure 5B As shown, the measurement engine 232 can determine the deviations 510 and 520 of multiple control segments located on the CAD drawing 505.

[0111] The measurement engine 232 can determine the pass / fail result of the part deviation at the control section by comparing the measured deviation with the tolerance associated with the control section. For Figure 8 In the example shown, measurement engine 232 can determine a pass / fail result for the deviation at control segment 810 based on a comparison of negative deviation 860 and the negative directional tolerance associated with control segment 810. If the measured deviation is within the associated tolerance, measurement engine 232 can determine the deviation result as "pass". In some embodiments, control unit 105 can provide a color-coded display on output device 110, for example, showing the determined deviation "pass" result value in light gray and the determined deviation "fail" result value in dark gray.

[0112] In some embodiments, the CAD drawing may include multiple control segments, and the measurement engine 232 may determine the overall deviation pass / fail result by combining the determined deviation pass / fail results of each control segment. For example, Figure 5A and Figure 5BThe determined overall deviation pass / fail results 530 and 540 are shown, respectively, based on the determined deviation pass / fail results of multiple control segments. Measurement engine 232 can determine the overall deviation pass / fail result as "pass" only if the determined deviation pass / fail result of each control segment is "pass". If the determined deviation pass / fail result of any control segment is "fail", measurement engine 232 can determine the overall deviation pass / fail result as "fail". In some embodiments, measurement engine 232 can determine the overall deviation pass / fail result as "pass" if the determined deviation pass / fail results of at least a threshold percentage of control segments are "pass". Control unit 105 can provide a color-coded display of the overall deviation pass / fail result on output device 110, for example, indicating an overall deviation "pass" result in light gray and indicating an overall deviation "fail" result in dark gray.

[0113] The overall deviation pass / fail result and the determined deviation pass / fail result for each control segment can depend on many factors, including the actual deviation of the part from its CAD drawing and the alignment of the CAD drawing with the image of the part. For example, Figure 5A and Figure 5B An example of the same CAD drawing superimposed on the same image of a patellofemoral joint implant is shown. Figure 5A In this process, the CAD drawing is overlaid on the image and roughly aligned, resulting in the determined deviations of multiple control segments (e.g., control segment 510) being "failed" and the overall deviation pass / fail result 530 being "failed". Figure 5B The image shows a CAD drawing superimposed on the image after alignment (e.g., automatic alignment via comparison engine 230), resulting in a "pass" result for the determined deviations of all control segments (e.g., including control segment 520) and an "pass" result for the overall deviation pass / fail result 540.

[0114] For reference Figure 9A Example 900 shows a portion 905 of a CAD drawing superimposed on a corrected and enhanced image of portion 910 of a hip implant. (Compared to...) Figure 3 , Figure 5A and Figure 5B In contrast to the patellofemoral joint implant shown, the hip joint implant may not be suitable for the field of view of imaging device 125. Figure 9AThe image of the hip implant shown does not include the entire hip implant. Part 905 of the CAD drawing can be aligned with part 910 of the hip implant visible in the image (e.g., automatically aligned via comparison engine 230). Measurement engine 232 can determine the deviation pass / fail result for each control segment included in part 905 of the CAD drawing (e.g., deviation pass / fail result 915). Measurement engine 232 can determine the overall deviation pass / fail result (e.g., overall deviation result 920 with "pass") by combining the determined deviation pass / fail results for each control segment.

[0115] For reference Figure 9B The superimposed layer is shown in the diagram. Figure 9A Another portion 960 of the hip implant shown in the corrected and enhanced image Figure 9A Example 950 shows another portion 955 of the CAD drawing. For parts like hip implants (which are not suitable for the field of view of imaging devices), if a single image is used to align the CAD drawing with the part, the alignment may be degraded at portions of the CAD drawing and at the parts furthest from the portion used for alignment. For example, Figure 9B It is shown that, with Figure 9A The alignment between parts 905 and 910 shown is worse than the alignment between part 955 in the CAD drawing and part 960 of the hip implant. Therefore, when multiple images are needed to capture all relevant features of a part and alignment is performed using a single image, the alignment of one or more additional images may deteriorate.

[0116] Figure 9B The degraded alignment shown may result in incorrect overall deviations leading to pass / fail results, similar to the reference above. Figure 5A The situation described. For Figure 9B In the example shown, measurement engine 232 can determine the pass / fail result 965 of multiple control segments as "failure" due to degraded alignment. Furthermore, measurement engine 232 can determine an overall deviation result 970 as "failure" based on the determined deviation results of the multiple "failures" of the control segments. The problem of erroneous overall deviation pass / fail results can be resolved by first performing alignment using multiple images and then determining the deviation pass / fail result using each of the multiple images.

[0117] For reference Figure 10AThe image 1010 shows the shape of a turbine blade root at 10x magnification. When determining the conformity of the turbine blade root shape, all features of interest can be fitted within the field of view of the imaging device. Therefore, the CAD drawing of the turbine blade root shape can be aligned with a single image at 10x magnification, and conformity with the CAD drawing can be determined. However, the turbine blade root shape may need to be manufactured with high precision, and higher levels of optical magnification may be required to check for compliance with tighter tolerance constraints. Figure 10B Image 1020 showing the shape of the turbine blade root at 20x magnification is shown, and Figure 10C Image 1030 shows the shape of a turbine blade root at 50x magnification. At higher magnifications, the field of view of the imaging device may shrink proportionally, and multiple images may be required to capture all the features of interest. (See above reference) Figure 9B Similar to the described scenario, if alignment is performed using only a single image, an incorrect overall deviation pass / fail result may be determined. The problem of incorrect overall deviation pass / fail results can be resolved by first performing alignment using multiple images, and then using each of those images to determine the deviation pass / fail result.

[0118] For reference Figure 11 The diagram illustrates a flowchart of an example method 1100 for aligning a CAD drawing with multiple images of a part according to the present disclosure. For example, method 1100 may be executed by control unit 105.

[0119] At 1105, multiple images of the part can be received. For example, control unit 105 can receive multiple images of part 120 from imaging device 125. Each of the multiple images can correspond to a part of the part, and the combination of the multiple images can provide an overall representation of the part. For example, the parts of the part may include portions 910 and 960 of a hip implant that are too large to fit within the field of view of the imaging device (respectively at...). Figure 9A and Figure 9B (As shown in the image). Multiple images can be acquired by the imaging device at multiple imaging locations. In another example, a portion of the part can be the shape of the turbine blade root at high magnification. Figure 10B or Figure 10C The portion shown is shown in the image. Each of the multiple images may include one or more features of interest for determining the conformity of the part. The combination of the multiple images may include all the features of interest required to determine the conformity of the part.

[0120] A CAD drawing may include one or more control segments. One or more control segments may include a single point at multiple locations. For example, the included control segment may be control segment 400. In some embodiments, one or more control segments may include multiple points located along the CAD entity, wherein the spacing between the multiple points is based on the resolution of multiple images. Each control segment may include a positive direction tolerance and a negative direction tolerance associated with the control segment. Each of the received multiple images may include a portion of a part corresponding to at least one of the one or more control segments.

[0121] Image correction and image enhancement operations can be performed on the received image. For example, image processing engine 226 can perform real-time image correction and image enhancement operations on the image stream received from imaging device 125.

[0122] At position 1110, multiple images and imaging position data associated with those images can be stored. The imaging position data for each image can indicate the relative position of the part with respect to the imaging device during imaging. For Figure 1 As shown in the example block diagram 100, control unit 105 can provide control commands to control the movement of platform 135b for mounting part 120. Multiple images can be acquired by imaging device 125 at different x, y, and z positions on platform 135b. Encoder 140 can be used to measure imaging position data indicating the relative position of part 120 with respect to imaging device 125 during imaging. Control unit 105 can receive multiple images from imaging device 125 and associated imaging position data from encoder 140. In some embodiments, control unit 105 can store multiple images and associated imaging position data in database 224.

[0123] At 1115, initial alignment of the CAD drawing with multiple images of the part can be performed. For example, comparison engine 230 can perform initial alignment of a portion of the CAD drawing with an image representing a portion of the part. Initial alignment can be performed in real time as multiple images are received. In some embodiments, initial alignment can be performed against stored images retrieved from database 224. Figure 5A Example 500 shows a CAD drawing 505 overlaid on a corrected and enhanced image of a patellofemoral joint implant and initially aligned. The initial alignment can be performed using a manual or automatic alignment process. As described above, the manual alignment process can be based on user input received at input devices 115a, 115b, and the automatic alignment process can be performed automatically using pattern matching.

[0124] At 1120, the initial overall alignment adjustment can be determined based on the initial alignment. For example, control unit 105 can determine translation and / or rotation adjustments of the CAD drawing corresponding to the initial alignment between the CAD drawing and the image of the part. For example, translation adjustments may include x-axis adjustments and / or y-axis adjustments, and rotation adjustments may include rotation angle adjustments in the xy plane. In some embodiments, for example, where multiple images include images acquired at different z-axis or depth positions, translation adjustments may also include z-axis adjustments, and rotation adjustments may include rotation angle adjustments in the yz plane and / or XZ plane.

[0125] At position 1125, alignment iteration can be performed. For example, control unit 105 can perform alignment iteration. Now refer to Figure 12 The diagram illustrates a flowchart of an example method 1200 for performing alignment iterations according to the present disclosure. For example, control unit 105 may execute method 1200 in conjunction with method 1100 (at 1125).

[0126] At 1205, for an image among multiple images, a portion of the CAD drawing corresponding to a part visible in that image can be determined. The corresponding portion of the CAD drawing can be determined based on imaging position data associated with the image. For example, control unit 105 can determine the difference in imaging position data between the current image and the image used for initial alignment at 1120. Control unit 105 can then use the initial alignment and the determined difference to determine the corresponding portion of the CAD drawing.

[0127] At 1210, it can be determined whether an alignment adjustment has been determined for the control segment included in the portion of the CAD drawing (determined at 12005) during the current alignment iteration. For example, control unit 105 can determine whether an alignment adjustment has been determined for the control segment during the current alignment iteration. If the alignment adjustment has not been determined, method 1200 can proceed to 1215. If the alignment adjustment has been determined, method 1200 can proceed directly to 12025.

[0128] In some embodiments, if an alignment adjustment has been determined and a skip parameter has been set, method 1200 can proceed to 1215. The skip parameter may be based on user input received at input devices 115a and 115b. In some embodiments, the skip parameter may be a value stored in database 224. For the example where an alignment adjustment has been determined and a skip parameter has been set, method 1200 may also include performing a weighted average of the alignment adjustment determined at 1220 and a previous alignment adjustment determined for the control segment during the current alignment iteration. For example, for three images out of a plurality of images, a specific control segment may be included. If a skip parameter is set, three alignment adjustments (corresponding to the three images in the alignment iteration) may be determined for that specific control segment, and a weighting factor of 1 / 3 may be applied to each of the three alignment adjustments. The weighting factors may not always be equal. Different weighting factors may be used depending on the control segment and the image in which the control segment is included.

[0129] At position 1215, the measurement point corresponding to the control position can be located on the image of the part. The measurement point can be located using pattern matching or a search range corresponding to the control position. For example, control unit 105 can use the method described in the reference above. Figures 6 to 7 The described search area is used to determine the measurement points. For example... Figure 6 As shown, the measurement engine 232 of the control unit 105 can determine the measurement point 645 based on the intersection of the search range 620 and the edge 640 of the part 605. If the measurement point is located, method 1200 can proceed to 1220.

[0130] However, if the initial alignment of the CAD drawing with multiple images of the part is inappropriate, the measurement point may not be located within the search range corresponding to the control segment. For example, after the initial alignment at 1115 of method 1100, the misalignment between the CAD drawing and the part may be greater than the search range corresponding to the control segment. Therefore, the search range may not intersect any edge of the part, and the measurement point corresponding to the control segment may not be determined using the search range. In this example, using a longer search range can help overcome this problem. However, in some cases, a longer search range may intersect with multiple edges of the part, thereby increasing the complexity of method 1200 and potentially leading to incorrect determination of the measurement point. Using a shorter search range can reduce the probability of intersecting with multiple edges of the part. However, using a shorter search range may require better initial alignment at 1115 of method 1100.

[0131] For reference Figure 13Example 1300 shows a CAD drawing 1305 superimposed on an image of part 1310. In example 1300, the initial alignment between the CAD drawing 1305 and part 1310 allows the search range 1315 to be closer to the edge 1320 of part 1310 than the edge 1325 of part 1310. Therefore, incorrect measurement points may be located on edge 1320 instead of the correct measurement points on edge 1325. In this example, a possible solution could be an additional control segment corresponding to the top edge of part 1310. This would better achieve the initial alignment of the CAD drawing and the part. Another possible solution could be to use the pattern matching procedure of program 222, as described above, to perform the initial alignment of the CAD drawing and the part.

[0132] Now back Figure 12 At point 1220, the alignment adjustment between the measurement point (located at point 1215 in method 1200) and the control segment can be determined. For example, control unit 105 can determine the alignment adjustment between the measurement point and the control segment. Now refer to Figure 14 The illustration 1400 shows a schematic representation 1400 of an example CAD drawing aligned with an image of part 1405. The CAD drawing may include a control segment 1410 at a nominal location 1415 on CAD entity 1425. The schematic representation 1400 includes a measurement point 1445 corresponding to the intersection of the search range of the control segment 1410 and the edge 1440 of part 1405. Alignment adjustments between the measurement point and the control segment may include translation and / or rotation adjustments.

[0133] Alignment adjustments can be determined based on a deviation nominal method. For example, measurement engine 232 can determine translation adjustments, including x-axis alignment adjustment 1460 and y-axis alignment adjustment 1465, based on the difference between measurement point 1445 and nominal position 1415. Measurement engine 232 can also determine the corresponding physical x-axis and y-axis alignment adjustments based on the calibration relationship between physical distance and the distance between a pair of pixels in an optical image. Furthermore, measurement engine 232 can determine rotational alignment adjustments.

[0134] For reference Figure 15The illustration 1500 shows a schematic representation 1500 of an example CAD drawing aligned with an image of a part. The CAD drawing may include control segments 1510 on CAD entity 1525. Schematic representation 1500 includes a measurement point 1545 corresponding to the intersection of the search range 1520 and the edge 1540 of the part. Schematic representation 1500 also includes a rotation center 1550 for determining rotational alignment adjustments. The rotation center 1550 may be set based on user input received at input devices 115a and 115b. In some embodiments, the rotation center 1550 may be automatically determined, for example, as the centroid of all control segments included in the determined portion of the CAD drawing. Figure 15 As shown, the measurement engine 232 can determine the rotation alignment adjustment 1570 based on the angles 1560 and 1565 corresponding to the control segment 1510 and the measurement point 1545, respectively.

[0135] In some embodiments, a multiplier can be applied to the determined rotation alignment adjustment to accelerate rotation convergence and reduce the number of alignment iterations required before a termination condition is met. For example, a multiplier of 1.1 can be applied to the determined rotation alignment adjustment. However, larger multiplier values ​​can lead to divergence and suboptimal alignment. In some embodiments, the control unit 105 may include the maximum value of the multiplier stored in the database 224. For example, the maximum value of the multiplier may be 1.2.

[0136] In some cases, the nominal deviation method can produce alignment results where parts in almost all parts of the CAD drawing are very closely aligned, but slightly exceed tolerances in a small portion. Users may wish to adjust the alignment to bring all parts of the CAD drawing into tolerance, even if this adjustment slightly degrades the alignment in other parts (while still remaining within tolerance). The described embodiments can achieve such alignment adjustments by using methods other than the nominal deviation method to determine the alignment adjustment.

[0137] For example, alignment adjustments can be determined using a fitting tolerance method. Now back to... Figure 6 Schematic representation 600 shows an example of a CAD drawing aligned with an image of part 605. Schematic representation 600 shows a control segment 610 located at nominal position 615 and a measuring point 645 located on the edge 640 of part 605. Schematic representation 600 also shows the positive tolerance 630 and negative tolerance 635 associated with the control segment 610. Figure 6 As shown, measurement point 645 may exceed the tolerance limit associated with control segment 610. Measurement engine 232 may determine one or more alignment adjustments to reduce the distance between measurement point 645 and negative tolerance 635.

[0138] In some embodiments, alignment adjustments can be determined using a combination of methods. For example, measurement engine 232 can use a deviation nominal method for the initial alignment iteration and then switch to using a fitting tolerance method based on a switching condition. The switching condition can be the number of alignment iterations or a threshold used for alignment adjustments.

[0139] Now back Figure 12 At 1225, it can be determined whether all control segments included in the CAD drawing have been processed during the current alignment iteration. If all control segments have been processed, method 1200 can proceed to 1230. If any remaining control segments exist, method 1200 can proceed to 1210 and determine whether an alignment adjustment has been determined for the next control segment during the current alignment iteration.

[0140] At 1230, it can be determined whether all images included in the multiple images have been processed during the current alignment iteration. If any images remain, method 1200 can proceed to 1205 and determine the portion of the CAD drawing corresponding to the portion of the part visible in the next image. If all images have been processed, method 1200 can proceed to 1235. For example, at 1235, method 1200 can proceed to 1130 of method 1100.

[0141] Now back Figure 11 At 1130, the overall alignment adjustment can be changed based on the alignment adjustment determined for the alignment iteration. For example, control unit 105 can change the overall alignment adjustment between the CAD drawing and the part by combining the alignment adjustments determined by method 1200 for each control segment included in multiple images. This can improve the alignment between the images of the CAD drawing and the part before performing the next alignment iteration. The overall alignment adjustment can be changed based on the overall translation adjustment and / or the overall rotation adjustment. For example, control unit 105 can determine the overall x-axis translation adjustment by adding an x-axis alignment adjustment 1460 for each control segment included in multiple images. Control unit 105 can determine the overall y-axis translation adjustment by adding a y-axis alignment adjustment 1465 for each control segment included in multiple images. Control unit 105 can determine the overall rotation adjustment by adding a rotation alignment adjustment 1570 for each control segment included in multiple images.

[0142] In embodiments that include one or more images corresponding to multiple z-axis imaging positions, the control unit 105 may perform the methods 1100 and 1200 described above, thereby further including translational alignment adjustments along the z-axis and rotational alignment adjustments in the yz plane and / or XZ plane.

[0143] At point 1135, it can be determined whether a termination condition for the alignment iteration is met. The termination condition may include a minimum total alignment adjustment threshold. For example, control unit 1135 may determine that the termination condition is met when one or both of the total translation adjustment and total rotation adjustment determined for the alignment iteration are below the corresponding threshold. The termination condition may be based on individual total translation adjustment thresholds in the x, y, and / or z-axis directions.

[0144] In some embodiments, the termination condition may include a maximum iteration threshold. For example, if the initial alignment between the CAD drawing and the part is inappropriate, the termination condition of a minimum total alignment adjustment threshold may still not be met even after a large number of iterations. In this case, including a termination condition with a maximum number of iterations can save computational resources by terminating the alignment iterations.

[0145] At 1140, an indication of the deviation of a part from its CAD drawing can be provided to the user on the output device 110. This indication can be based on the alignment between the CAD drawing and multiple images. For example, after applying the final value of the overall alignment adjustment to the CAD drawing, the control unit 105 can determine the deviation of the part from its CAD drawing at each control segment included in the multiple images. (Refer to the above...) Figure 8 As explained, the measurement engine 232 can determine the deviation 860 of part 805 from its CAD drawing at control segment 810. Control unit 105 can use the same control segment and multiple images to determine the deviation, which are aligned using method 1100. No new images are required to determine the deviation. This improves the speed and efficiency of the described embodiment. For each control segment of the multiple images, control unit 105 can verify whether the part is within the tolerance zone associated with that control segment and provide an indication, for example, on output device 110. The indication may include an overall deviation pass / fail result, such as as referenced above. Figure 5B The overall deviation described is expressed in result 540.

[0146] The embodiments described above provide examples of aligning a CAD drawing with one or more images of a part. Those skilled in the art will recognize that in some embodiments, alignment can be performed by aligning one or more images of the part with the CAD drawing. In some embodiments, alignment between the CAD drawing and the images of the part can be performed by controlling the relative motion between the part and an imaging device used to acquire the images. For example, controlled relative motion can be achieved using the methods described above. Figure 1 Platform 135b and / or platform 135c shown are provided.

[0147] It will be apparent to those skilled in the art that certain adjustments and modifications can be made to the described method, and the embodiments discussed above for aligning multiple images of a part with a CAD drawing should be considered illustrative rather than limiting.

[0148] While the foregoing description describes features of exemplary embodiments, it will be understood that certain features and / or functions of the described embodiments are readily modified without departing from the spirit and operating principles of the described embodiments. For example, various features described by means of the illustrated embodiments or examples may be selectively combined with each other. In other instances, well-known methods, processes, and components have not been described in detail so as not to obscure the description of the embodiments. Therefore, what has been described above is intended to illustrate the claimed concepts and is not restrictive. Those skilled in the art will understand that other variations and modifications can be made without departing from the scope of the invention as defined in the appended claims. The scope of the claims should not be limited by the preferred embodiments and examples, but should be given the broadest interpretation consistent with the overall description.

Claims

1. A system for aligning and comparing a CAD drawing including one or more control segments with multiple images of a part, wherein each of the multiple images corresponds to a portion of the part, and the combination of the multiple images provides an overall representation of the part, the system comprising: An imaging device for acquiring the plurality of images, wherein each of the plurality of images corresponds to at least one of the one or more control segments; A memory module for storing the plurality of images and imaging position data associated with the plurality of images, wherein the imaging position data for each image indicates the relative position of the part with respect to the imaging device during imaging; The processing module is configured to perform initial alignment of the CAD drawing with the plurality of images of the part, determine an initial overall alignment adjustment based on the initial alignment, and perform one or more alignment iterations until a termination condition is met, wherein each alignment iteration includes: For each of the plurality of images of the part: Identify the portion of the CAD drawing corresponding to the portion of the part visible in the image of the part, and for each control segment included in the portion of the CAD drawing: If the alignment adjustment corresponding to the control segment has not been determined during the alignment iteration, then a measurement point is located on the image of the part corresponding to the control segment; and Determine the alignment adjustment between the measurement point and the control segment; The total alignment adjustment is changed based on the alignment adjustment determined for this alignment iteration; and Determine whether the termination condition is met; and An output device for providing a user with an indication of deviation based on the alignment between the CAD drawing and the plurality of images.

2. The system of claim 1, wherein the one or more control segments comprise a single point at a plurality of locations.

3. The system of claim 1, wherein the one or more control segments comprise a plurality of points located along a CAD entity, wherein the spacing between the plurality of points is based on the resolution of the plurality of images.

4. The system of claim 1, wherein the initial alignment is based on user input received at the input device.

5. The system of claim 1, wherein the initial alignment is performed automatically using pattern matching.

6. The system of claim 1, wherein the measuring point is located using pattern matching.

7. The system of claim 1, wherein the measurement point is located using a search range corresponding to the control segment.

8. The system of claim 1, wherein if the alignment adjustment corresponding to the control segment has been determined and a skip parameter has been set during the alignment iteration, the processing module is further configured to perform a weighted average of the plurality of alignment adjustments corresponding to the control segment.

9. The system of claim 1, wherein the alignment adjustment between the measurement point and the control segment is determined based on a deviation nominal method.

10. The system of claim 1, wherein the alignment adjustment between the measurement point and the control segment is determined based on a fitting tolerance method.

11. The system of claim 1, wherein the total alignment adjustment includes translation adjustment and / or rotation adjustment.

12. The system of claim 1, wherein the alignment iteration termination condition includes a maximum iteration threshold and a minimum total alignment adjustment threshold.

13. The system of claim 1, wherein providing, for the user, an indication of the bias based on the alignment comprises: For each control segment, verify whether the part is within the tolerance zone associated with that control segment.

14. A method for aligning and comparing a CAD drawing including one or more control segments with a plurality of images of a part, wherein each of the plurality of images corresponds to a portion of the part, and the combination of the plurality of images provides an overall representation of the part, the method comprising: Receive the plurality of images, wherein each of the plurality of images corresponds to at least one of the one or more control segments; The plurality of images and imaging position data associated with the plurality of images are stored, wherein the imaging position data for each image indicates the relative position of the part with respect to the imaging device during imaging; Perform initial alignment of the CAD drawing with the plurality of images of the part; The initial total alignment adjustment is determined based on the initial alignment. Perform one or more alignment iterations until a termination condition is met, where each alignment iteration includes: For each of the plurality of images of the part: Identify the portion of the CAD drawing corresponding to the portion of the part visible in the image of the part, and for each control segment included in the portion of the CAD drawing: If the alignment adjustment corresponding to the control segment has not been determined during the alignment iteration, then a measurement point is located on the image of the part corresponding to the control segment; and Determine the alignment adjustment between the measurement point and the control segment; The total alignment adjustment is changed based on the alignment adjustment determined for this alignment iteration; and Determine whether the termination condition is met; and The output device provides the user with an indication of the deviation, wherein the indication is based on the alignment between the CAD drawing and the plurality of images.

15. The method of claim 14, wherein the one or more control segments comprise a single point at a plurality of locations.

16. The method of claim 14, wherein the one or more control segments comprise a plurality of points located along a CAD entity, wherein the spacing between the plurality of points is based on the resolution of the plurality of images.

17. The method of claim 14, wherein the initial alignment is based on user input received at the input device.

18. The method of claim 14, wherein performing the initial alignment includes automatically performing the initial alignment using pattern matching.

19. The method of claim 14, wherein locating the measurement point includes using pattern matching.

20. The method of claim 14, wherein locating the measurement point includes using a search range corresponding to the control segment.

21. The method of claim 14, wherein if the alignment adjustment corresponding to the control segment has been determined during the alignment iteration and a skip parameter has been set, the method further includes performing a weighted average of the plurality of alignment adjustments corresponding to the control segment.

22. The method of claim 14, wherein determining the alignment adjustment between the measurement point and the control segment comprises determining the alignment adjustment based on a deviation nominal method.

23. The method of claim 14, wherein determining the alignment adjustment between the measurement point and the control segment comprises determining the alignment adjustment based on a fitting tolerance method.

24. The method of claim 14, wherein the total alignment adjustment includes translation adjustment and / or rotation adjustment.

25. The method of claim 14, wherein the alignment iteration termination condition includes a maximum iteration threshold and a minimum total alignment adjustment threshold.

26. The method of claim 14, wherein providing the user with an indication of the bias comprises: For each control segment, verify whether the part is within the tolerance zone associated with that control segment.

Images