Method for Automatically Generating Common Measurements across Multiple Assembly Units

Abstract

This application relates to a method for automatically generating a common measurement across multiple assembly units. A method includes: displaying a first image of a first assembly unit within a user interface; positioning a first virtual origin at a first feature on the first assembly unit; in response to a change in a view window of the first image, displaying a first sub-region of the first image within the user interface; recording the geometry and position of the first sub-region relative to the first virtual origin; positioning a second virtual origin at a second feature on a second assembly unit represented in a second image that is similar to the first feature; projecting the geometry and position of the first sub-region onto the second image according to the second virtual origin to define a second sub-region of the second image; and in response to receiving a command to advance from the first image to the second image, displaying the second sub-region within the user interface.

Description

[0001] This application is a divisional application of the application with the filing date of January 16, 2017, the application number of 201780010809.4, and the invention title of "Method for Automatically Generating Common Measurements across Multiple Assembly Units".

[0002] Cross - reference to related applications

[0003] This application claims the benefit of U.S. Provisional Application No. 62 / 279,174, filed on January 15, 2016, which is hereby incorporated by reference in its entirety. Technical field

[0004] The present invention generally relates to the field of optical inspection, and more particularly to new and useful methods for automatically generating common measurements across multiple assembly units in the field of optical inspection.

[0005] Aspects of the present disclosure may be implemented in one or more of the embodiments below.

[0006] 1) A method, comprising:

[0007] Displaying a first image of a first assembly unit within a user interface, the form of the first image being recorded at an optical inspection station;

[0008] Positioning a first virtual origin at a first feature on the first assembly unit represented in the first image;

[0009] In response to a change in a view window of the first image at the user interface, displaying a first sub - region of the first image within the user interface;

[0010] Recording the geometry and position of the first sub - region of the first image relative to the first virtual origin;

[0011] In response to receiving a command at the user interface to advance from the first image to a second image:

[0012] Positioning a second virtual origin at a second feature on a second assembly unit represented in the second image, the second feature on the second assembly unit being similar to the first feature on the first assembly unit;

[0013] Projecting the geometry and position of the first sub - region of the first image onto the second image according to the second virtual origin to define a second sub - region of the second image; and

[0014] Displaying the second sub - region of the second image within the user interface.

[0015] 2) According to the method of 1), wherein, displaying the first image of the first assembly unit includes:

[0016] Obtaining a first digital photographic image from a database, the first digital photographic image being recorded by the optical inspection station at a first time during the assembly;

[0017] Normalizing the first digital photographic image based on a reference image recorded at the optical inspection station to generate the first image; and

[0018] Providing the first image to a computing device executing the user interface for reproduction.

[0019] 3) According to the method of 2), further comprising:

[0020] Obtaining a second digital photographic image from the database, the second digital photographic image being recorded by the optical inspection station at a second time during the assembly; and

[0021] Normalizing the second digital photographic image based on the reference image to generate the second image.

[0022] 4) According to the method of 1), wherein positioning the first virtual origin at the first feature on the first assembly unit represented in the first image includes, in response to a change in the view window of the first image at the user interface:

[0023] Identifying a set of discrete surfaces on the first assembly unit represented within a first sub-region of the first image;

[0024] Selecting a first discrete surface within the set of discrete surfaces that exhibits the largest dimension;

[0025] Identifying the first feature that demarcates the first discrete surface; and

[0026] Positioning the first virtual origin at the first feature in the first image.

[0027] 5) According to the method of 1), wherein positioning the second virtual origin at the second feature on the second assembly unit represented in the second image includes:

[0028] Projecting a boundary that encloses and is offset from the first virtual origin onto the second image;

[0029] Identifying a set of edge features on the second assembly unit represented within a region of the second image contained within the boundary; and

[0030] Identify a second feature in the set of edge features that exhibits a second geometry similar to the first geometry of the first feature; and

[0031] Locate the second virtual origin on the second feature.

[0032] 6) The method according to 1):

[0033] Wherein, locating the first virtual origin on the first feature on the first assembly unit represented in the first image includes locating the first virtual origin of the first image on the first feature that defines a first corner of a first part in the first assembly unit; and

[0034] Wherein, locating the second virtual origin on the second feature on the second assembly unit represented in the second image includes:[[]]

[0035] Identifying a set of parts in the second assembly unit represented in the second image;

[0036] Selecting a second part similar to the first part from the set of parts; and

[0037] Locating the second virtual origin of the second image on the second feature that defines a second corner of the second part.

[0038] 7) The method according to 1):

[0039] Wherein, displaying the first sub-region of the first image within the user interface includes displaying the first sub-region of the first image within the user interface in response to a change in the view window of the first image at a first time;

[0040] Wherein, locating the first virtual origin in the first image on the first feature on the first assembly unit represented in the first image includes, in response to a change in the view window of the first image:[[]]

[0041] Identifying a set of edge features on the first assembly unit within the first sub-region of the first image;

[0042] At a second time after the first time, receiving a selection of a pixel within the first sub-region of the first image;

[0043] Selecting the first feature closest to the pixel from the set of edge features; and

[0044] Locating the first virtual origin on the first feature.

[0045] 8) The method according to 1), wherein displaying the first sub-region of the first image within the user interface includes displaying the first sub-region of the first image within the user interface in response to a change in the view window of the first image, the change including a zoom input for the first image.

[0046] 9) The method according to 8):

[0047] wherein recording the geometry of the first sub-region of the first image relative to the first virtual origin includes recording the zoom level selected for the first sub-region of the first image;

[0048] wherein recording the position of the first sub-region of the first image relative to the first virtual origin includes recording the vertical offset and the horizontal offset between the first virtual origin in the first image and the origin of the first sub-region; and

[0049] wherein projecting the geometry and the position of the first sub-region of the first image onto the second image according to the second virtual origin to define the second sub-region of the second image includes:

[0050] defining the geometry of a second view window according to the zoom level;

[0051] vertically offsetting the origin of the second view window from the second virtual origin within the second image according to the vertical offset;

[0052] horizontally offsetting the origin of the second view window from the second virtual origin within the second image according to the horizontal offset; and

[0053] defining the region of the second image delimited by the second view window as the second sub-region.

[0054] 10) The method according to 9):

[0055] wherein positioning the first virtual origin in the first image at the first feature on the first assembly unit represented in the first image further includes:

[0056] positioning the first virtual origin of a first coordinate system on the first feature within the first image;

[0057] aligning the first axis of the first coordinate system with the first feature;

[0058] wherein recording the geometry and the position of the first sub-region of the first image relative to the first virtual origin further includes:

[0059] An angular offset recorded between an edge of the first sub-region of the first image and the first axis of the first coordinate system; and

[0060] wherein positioning the second virtual origin of the second coordinate system on the second feature on the second assembly unit represented in the second image includes:

[0061] positioning the second virtual origin of the second coordinate system on the second feature within the second image; and

[0062] aligning a second axis of the second coordinate system with the second feature; and

[0063] wherein projecting the geometry and the position of the first sub-region of the first image onto the second image according to the second virtual origin to define a second sub-region of the second image further includes:

[0064] angularly offsetting an edge of the second sub-region from the second axis of the second coordinate system according to the angular offset.

[0065] 11) The method according to 1), further comprising:

[0066] providing, via the user interface, a prompt for confirming a projected geometry and a projected position of the second sub-region of the second image;

[0067] in response to receiving a confirmation of the projected geometry and the projected position of the second sub-region of the second image:

[0068] obtaining a set of images of the assembly unit from a database;

[0069] positioning a virtual origin in each image of the set of images; and

[0070] projecting the geometry and the position of the first sub-region of the first image onto each image of the set of images to define a set of sub-regions of the set of images; and

[0071] panning through the entire set of sub-regions displayed within the user interface in response to a scroll input at the user interface.

[0072] 12) The method according to 1), further comprising:

[0073] setting a first opacity of the first sub-region of the first image;

[0074] setting a second opacity of the second sub-region of the second image;

[0075] Overlay the second sub-region on the first sub-region to generate a composite image;

[0076] Display the composite image within the user interface;

[0077] Adjust the first opacity according to an input received at the user interface;

[0078] Adjust the second opacity as an inverse function of the first opacity; and

[0079] Refresh the composite image in response to the input.

[0080] 13) The method according to 1), further comprising:

[0081] Obtain a virtual computer-aided drafting model representing the first assembly unit;

[0082] Generate a computer-aided drafting image of the virtual computer-aided drafting model at an orientation and perspective approximately the same as the orientation and position of the first assembly unit represented in the first image;

[0083] Locate a third virtual origin at a third feature in the computer-aided drafting image, the third feature being similar to the first feature on the first assembly unit;

[0084] Project the geometry and position of the first sub-region of the first image onto the virtual computer-aided drafting model according to the third virtual origin to define a third image; and

[0085] Display the third image in a translucent form on the first sub-region of the first image within the user interface.

[0086] 14) A method, comprising:

[0087] Display, within a user interface, a first image of an assembly unit in a first assembly stage, the first image being recorded at a first optical inspection station;

[0088] Locate a first virtual origin in the first image at a feature on the assembly unit represented in the first image;

[0089] In response to a zoom input received at the user interface, display a first sub-region of the first image within the user interface;

[0090] Store the geometry and position of the first sub-region of the first image relative to the first virtual origin;

[0091] Identify the feature on the assembly unit in a second image of the assembly unit in the second assembly stage;

[0092] Locate a second virtual origin in the second image based on the feature;

[0093] Define a second sub-region of the second image based on the geometry and position of the first sub-region of the first image and the second virtual origin; and

[0094] In response to receiving a command to advance from the first image to the second image at the user interface, display the second sub-region of the second image within the user interface.

[0095] 15) The method according to 14):

[0096] Wherein, displaying the first image of the assembly unit in the first assembly stage includes:

[0097] Obtain a first digital photographic image from a database, the first digital photographic image being recorded by the first optical inspection station arranged at a first position along the assembly line;

[0098] Normalize the first digital photographic image based on a first reference image recorded at the first optical inspection station to generate the first image; and

[0099] Provide the first image to a computing device executing the user interface for display within the user interface; and

[0100] Wherein, displaying the second sub-region of the second image within the user interface includes:

[0101] Obtain a second digital photographic image from the database, the second digital photographic image being recorded by the second optical inspection station arranged at a second position along the assembly line; and

[0102] Normalize the second digital photographic image based on a second reference image recorded at the second optical inspection station to generate the second image.

[0103] 16) The method according to 14), further comprising:

[0104] Set a first opacity of the first sub-region of the first image;

[0105] Set a second opacity of the second sub-region of the second image;

[0106] Combine the first sub-region and the second sub-region into a composite image; and

[0107] Display the composite image within the user interface.

[0108] 17) The method according to 16) further comprises:

[0109] Position a third virtual origin in the second image at a second feature on the assembly unit represented in the second image;

[0110] Store the geometry and position of the second sub-region of the second image relative to the third virtual origin;

[0111] Identify the second feature on the assembly unit in a third image;

[0112] Locate a fourth virtual origin in the third image based on the second feature;

[0113] Define a third sub-region of the third image based on the geometry and the position of the second sub-region of the second image and the fourth virtual origin; and

[0114] Insert the third sub-region into the composite image.

[0115] 18) A method comprising:

[0116] Display a first image of a first assembly unit within a user interface, the first image being recorded at an optical inspection station;

[0117] In response to a change in a view window of the first image at the user interface, display a first sub-region of the first image within the user interface;

[0118] Record the geometry and position of the first sub-region of the first image relative to a first feature represented in the first image;

[0119] Identify a second feature represented in a second image of a second assembly unit, the second feature being similar to the first feature;

[0120] Project the geometry and the position of the first sub-region of the first image relative to the first feature onto the second image based on the second feature to define a second sub-region of the second image; and

[0121] In response to receiving a command at the user interface to advance from the first image to the second image, display the second sub-region of the second image within the user interface in place of the first image.

[0122] 19) The method according to 18):

[0123] It further includes a first virtual origin for positioning the first image on the first feature, and the first feature defines a reference point for holding the first assembly unit on a fixture in the optical inspection station;

[0124] Wherein, recording the geometry and position of the first sub-region of the first image relative to the first feature includes recording the geometry and position of the first sub-region of the first image relative to the first virtual origin;

[0125] Wherein, identifying a second feature represented in a second image of a second assembly unit includes:

[0126] Identifying a set of features represented in the second image;

[0127] Selecting, from the set of features, the second feature that exhibits dimensional and geometric features approximating those of the first feature; and

[0128] Positioning a second virtual origin of the second image on the second feature that defines a second corner of a second part; and

[0129] Wherein, projecting the geometry and position of the first sub-region of the first image relative to the first feature onto the second image includes projecting the geometry and position of the first sub-region of the first image relative to the first feature onto the second image according to the second virtual origin to define a second sub-region of the second image.

[0130] 20) The method according to 18):

[0131] Wherein, recording the geometry and position of the first sub-region of the first image relative to the first feature represented in the first image includes, in response to a change in a view window of the first image at the user interface:

[0132] Identifying a set of discrete surfaces on the first assembly unit represented within the first sub-region of the first image;

[0133] Selecting, from the set of discrete surfaces, a first discrete surface that exhibits the largest dimension;

[0134] Identifying the first feature that delimits the first discrete surface; and

[0135] Positioning the boundary of the first sub-region of the first image relative to the first feature in the first image.

[0136] 21) The method according to 18), wherein recording the geometry and the position of the first sub-region of the first image relative to the first feature represented in the first image includes:

[0137] Projecting a boundary that encloses the first feature and is offset from the first feature onto the second image;

[0138] Identifying a set of edge features represented in a region of the second image that is contained within the boundary; and

[0139] Identifying a second feature in the set of edge features that exhibits a second geometry that is approximate to the first geometry of the first feature.

[0140] 22) A method for automatically generating common measurements across multiple assembly units, comprising:

[0141] Displaying a first image within a user interface, the form of the first image being recorded at an optical inspection station;

[0142] Receiving a manual selection of a specific feature in a first assembly unit represented in the first image;

[0143] Receiving a selection of a measurement type for the specific feature;

[0144] Extracting a first true dimension of the specific feature in the first assembly unit from the first image according to the measurement type;

[0145] For each image in a set of images:

[0146] Identifying a feature in an assembly unit represented in the image, the feature in the assembly unit being similar to the specific feature in the first assembly unit; and

[0147] Extracting a true dimension of the feature in the assembly unit from the image according to the measurement type; and

[0148] Aggregating the first true dimension and a set of true dimensions extracted from the set of images into a digital container.

[0149] 23) The method according to 22):

[0150] Wherein, displaying the first image includes:

[0151] Obtaining a first digital photographic image from a database, the first digital photographic image being recorded by the optical inspection station at a first time during assembly;

[0152] Normalize the first digital photographic image based on a reference image recorded at the optical inspection station to generate the first image; and

[0153] Provide the first image to a computing device that executes the user interface for reproduction;

[0154] Wherein, extracting the first true size of the specific feature in the first assembly unit from the first image includes:

[0155] Project a dimensional space onto the first image; and

[0156] Extract the first true size of the specific feature from the first image based on the position of the specific feature relative to the dimensional space and the measurement type.

[0157] 24) The method according to 23):

[0158] Further comprising:

[0159] Obtain a second digital photographic image from the database, the second digital photographic image being recorded by the optical inspection station at a second time during the assembly; and

[0160] Normalize the second digital photographic image based on the reference image to generate a second image in the set of images;

[0161] Wherein, for each image in the set of images, identifying a feature in the assembly unit represented in the image includes identifying a second feature in a second assembly unit represented in the second image, the second feature in the second assembly unit being similar to the specific feature in the first assembly unit; and

[0162] Wherein, for each image in the set of images, extracting the true size of a feature in the assembly unit from the image includes:

[0163] Project the dimensional space onto the second image; and

[0164] Extract the second true size of the second feature from the second image based on the position of the second feature relative to the dimensional space and the measurement type.

[0165] 25) The method according to 22):

[0166] Wherein, displaying the first image within the user interface includes:

[0167] Identify a first set of features in the first image;

[0168] Generate a first feature space, the first feature space including a first set of vectors representing the first set of features; and

[0169] Display the first feature space within the user interface together with the first image;

[0170] Wherein, receiving a manual selection of the specific feature in the first assembly unit represented in the first image includes:

[0171] Receiving a manual selection of a specific vector feature from the first set of vectors included in the first feature space; and

[0172] Identifying the specific feature corresponding to the specific vector;

[0173] Wherein, identifying the features in the assembly unit represented in the image for each image in the set of images includes, for each image in the set of images:

[0174] Identifying a set of features in the image;

[0175] Generating a feature space, the feature space including a set of vectors representing the set of features;

[0176] Aligning the feature space with the first feature space;

[0177] Identifying the vector in the set of vectors that is closest in position and geometry to the specific vector in the first set of vectors; and

[0178] Marking the feature in the image corresponding to the vector as similar to the specific feature.

[0179] 26) The method according to 25):

[0180] Wherein, identifying the first set of features in the first image includes identifying a set of surfaces, edges, and corners on the first assembly unit represented in the first image; and

[0181] Wherein, receiving a selection of the measurement type of the specific feature includes receiving a selection from a set of measurement types, the set of measurement types including: corner-to-corner distance, edge length, area, radius, and contour.

[0182] 27) The method according to 22), wherein, receiving a manual selection of the specific feature in the first assembly unit represented in the first image includes:

[0183] Receiving a manual selection of a specific pixel from the first image;

[0184] Identifying a first set of features in the first image; and

[0185] Identify the specific feature in the first set of features that is closest to the specific pixel.

[0186] 28) The method according to 22), wherein identifying the features in the assembly unit represented in the image for each image in the set of images comprises:

[0187] Defining a feature window that encloses the specific feature, deviates from the specific feature, and is positioned according to the global origin of the first image; and

[0188] For each image in the set of images:

[0189] Position the feature window within the image according to the global origin of the image; and

[0190] Identify the feature contained in the feature window as similar to the specific feature.

[0191] 29) The method according to 22), further comprising:

[0192] Identifying the assembly state of the first assembly unit; and

[0193] Selecting, from the body of images of a plurality of assembly units recorded across a set of optical inspection stations and representing the assembly units in various assembly states, the set of images representing the set of assembly units in the assembly state.

[0194] 30) The method according to 22), further comprising:

[0195] Identifying the serial number of the first assembly unit; and

[0196] Selecting, from the body of images of a plurality of assembly units recorded across a set of optical inspection stations and representing the assembly units in various assembly states, the set of images representing the first assembly unit in various assembly stages.

[0197] 31) The method according to 22):

[0198] Wherein aggregating the first true size and the set of true sizes extracted from the set of images into the digital container comprises aggregating the first true size and the set of true sizes into a virtual histogram that includes a set of discrete percentile ranges spanning the first true size and the set of true sizes; and

[0199] The method further comprises:

[0200] Reproducing the virtual histogram within the user interface; and

[0201] In response to a selection of a particular percentile range within the set of discrete percentile ranges, a particular image representing the particular percentile range is reproduced within the user interface.

[0202] 32) The method according to 22), further comprising:

[0203] Accessing a dimensional range of a feature similar to the particular feature associated with a failure of a component;

[0204] Identifying a second assembly unit, the second assembly unit including a second feature similar to the particular feature and characterized by a second true dimension included within the dimensional range, the second assembly unit being represented in a second image of the set of images; and

[0205] Providing a prompt to inspect the second assembly unit to an electronic account associated with a user.

[0206] 33) The method according to 22), wherein aggregating the first true dimension and the set of true dimensions extracted from the set of images into the digital container includes:

[0207] Calculating a true dimension range spanning the first true dimension and the set of true dimensions extracted from the set of images;

[0208] Selecting, in the set of images, a second image representing a second assembly unit, the second assembly unit including a second feature similar to the particular feature and characterized by a second dimension close to a first end of the true dimension range;

[0209] Selecting, in the set of images, a third image representing a third assembly unit, the third assembly unit including a third feature similar to the particular feature and characterized by a third dimension close to a second end of the true dimension range;

[0210] Generating a composite image including the second image and the third image overlaid on the second image; and

[0211] Reproducing the composite image within the user interface.

[0212] 34) The method according to 22):

[0213] Further comprising:

[0214] Receiving a manual selection of a second particular feature in the first assembly unit represented in the first image;

[0215] Receiving a selection of a second measurement type for the second particular feature;

[0216] Extract a second true dimension of the second specific feature in the first assembly unit from the first image according to the second measurement type; and

[0217] For each image in the set of images:

[0218] Identify a second feature in the assembly unit represented in the image, the second feature in the assembly unit being similar to the second specific feature in the first assembly unit; and

[0219] Extract a second true dimension of the second feature in the assembly unit from the image according to the second measurement type; and

[0220] Wherein, aggregating the set of true dimensions extracted from the set of images into the digital container includes:

[0221] Fill a two-dimensional graph with points representing the measurement type spanning the first image and the set of images and the second measurement type spanning the first image and the set of images.

[0222] 35) The method according to 22), further comprising:

[0223] Generate a measurement specification that defines the first measurement type and characterizes the specific feature;

[0224] Receive a subscription to the measurement specification from a user; and

[0225] Distribute the digital container to an electronic account associated with the user based on the subscription.

[0226] 36) The method according to 22), further comprising:

[0227] Access the target dimension of the specific feature;

[0228] Access the dimensional tolerance of the target dimension of the specific feature; and

[0229] Mark the serial number of a second assembly unit, the second assembly unit including a second feature characterized by a true dimension that differs from the target dimension by more than the dimensional tolerance, the second unit being represented in a second image in the set of images, the second feature being similar to the specific feature.

[0230] 37) The method according to 36), wherein accessing the target dimension of the specific feature includes obtaining the target dimension from a computer-aided drafting model of the first assembly unit.

[0231] 38) The method according to 22), further comprising:

[0232] Defining an order of the set of images based on a true size of a feature similar to the specific feature extracted from an image in the set of images;

[0233] Substantially aligning an image in the set of images by a feature similar to the specific feature; and

[0234] In response to a scroll input at the user interface, transposing within the user interface according to the order throughout a reproduction of the images in the set of images.

[0235] 39) A method, comprising:

[0236] Obtaining a set of images;

[0237] For a first image in the set of images:

[0238] Displaying the first image within a user interface, the first image being in a form recorded at an optical inspection station;

[0239] Receiving a manual selection of a specific feature in a first assembly unit represented in the first image;

[0240] Determining a measurement type of the specific feature;

[0241] Extracting a first true size related to the specific feature in the first assembly unit from the first image according to the measurement type; and

[0242] Displaying the first true size within the user interface together with the first image;

[0243] For a second image in the set of images:

[0244] Automatically identifying a second feature in a second assembly unit represented in the second image, the second feature in the second assembly unit being similar to the specific feature in the first assembly unit; and

[0245] Extracting a second true size related to the second feature in the second assembly unit from the second image according to the measurement type; and

[0246] For a third image in the set of images:

[0247] Automatically identifying a third feature in a third assembly unit represented in the third image, the third feature in the third assembly unit being similar to the specific feature in the first assembly unit; and

[0248] Extracting a third true size related to the third feature in the third assembly unit from the third image according to the measurement type;

[0249] In response to the selection of the second image at the user interface, display the second image and the second true size within the user interface; and

[0250] In response to the selection of the third image at the user interface, display the third image and the third true size within the user interface.

[0251] 40) The method according to 39):

[0252] Wherein, automatically identifying the second feature in the second assembly unit represented in the second image includes executing a feature selection routine to identify the second feature in the second image;

[0253] The method further includes, in response to automatically identifying the second feature in the second assembly unit represented in the second image:

[0254] Display the second image within the user interface;

[0255] Indicate the second feature within the second image; and

[0256] Wherein, automatically identifying the third feature in the third assembly unit represented in the third image includes:

[0257] In response to receiving a manual confirmation at the user interface that the second feature is similar to the specific feature, identify the third feature in the third image according to the feature selection routine.

[0258] 41) The method according to 39):

[0259] Wherein, receiving the selection of the specific feature in the first assembly unit includes receiving the selection of the specific feature in the first assembly unit at a first time;

[0260] Wherein, identifying the second feature in the second assembly unit represented in the second image includes identifying the second feature in the second assembly unit represented in the second image recorded at the assembly line before the first time; and

[0261] The method further includes:

[0262] Access the target size of the specific feature;

[0263] Access the dimensional tolerance of the target size of the specific feature;

[0264] In response to receiving a fourth image recorded at a second time after the first time, identify a fourth feature in a fourth assembly unit represented in the fourth image, the fourth feature in the fourth assembly unit being similar to the specific feature in the first assembly unit;

[0265] Extract a fourth true size of the fourth feature in the fourth assembly unit from the fourth image according to the measurement type; and

[0266] In response to the fourth true size differing from the target size by more than the size tolerance, mark the fourth assembly unit. Brief Description of the Drawings

[0267] Figure 1 is a flowchart representation of a first method;

[0268] Figure 2 is a graphical representation of a variation of the first method;

[0269] Figure 3 is a flowchart representation of a second method;

[0270] Figure 4 is a flowchart representation of a variation of the second method;

[0271] Figure 5 is a graphical representation of a variation of the second method;

[0272] Figure 6 is a flowchart representation of a third method;

[0273] Figure 7 is a graphical representation of a variation of the third method;

[0274] Figure 8 is a graphical representation of a variation of the third method; and

[0275] Figure 9A 、 9B and 9C are graphical representations of variations of the third method.

[0276] Description of Embodiments

[0277] The following description of embodiments of the present invention is not intended to limit the present invention to these embodiments, but rather to enable those skilled in the art to make and use the present invention more precisely. The variations, configurations, implementations, example implementations, and examples described herein are optional, and the variations, configurations, implementations, example implementations, and examples described thereof are not exclusive. The present invention described herein may include any and all permutations of these variations, configurations, implementations, example implementations, and examples.

[0278] 1. Assembly Line Configuration

[0279] As Figure 1 shown, a first method S100 for automatically configuring an optical inspection along an assembly line includes: obtaining, in block S111, a first image captured by a first optical inspection station at a first time, the first image being associated with an identifier of the first optical inspection station, and a first timestamp corresponding to the first time; obtaining, in block S112, a second image captured by a second optical inspection station at a second time after the first time, the second image being associated with an identifier of the second optical inspection station, and a second timestamp corresponding to the second time; obtaining, in block S113, a third image captured by the first optical inspection station at a third time after the first time, the third image being associated with an identifier of the first optical inspection station, and a third timestamp corresponding to the third time; identifying, in block S121, a first serial number of a first assembly unit in the first image; identifying, in block S122, the first serial number in the second image; identifying, in block S123, a second serial number of a second assembly unit in the third image; determining, in block S130, positions of the first optical inspection station and the second optical inspection station along the assembly line based on the first timestamp before the second timestamp and the identification of the first serial number in the first image and the second image; determining, in block S140, positions of the first assembly unit and the second assembly unit along the assembly line at a specific time based on the identification of the first serial number in the second image associated with the second timestamp and the identification of the second serial number in the third image associated with the third timestamp; rendering, within a user interface, in block S150, a virtual representation of the assembly line and virtual representations of the first assembly unit and the second assembly unit along the assembly line at the specific time based on the determined positions of the first optical inspection station and the second optical inspection station and the determined positions of the first assembly unit and the second assembly unit along the assembly line at the specific time.

[0280] 1.1 Application

[0281] Generally, a production verification system (hereinafter referred to as the "system") can execute blocks of a first method S100 to automatically configure multiple optical inspection stations along an assembly line after the assembly line and the optical inspection stations are installed and after images of units passing through the assembly line are imaged by the optical inspection stations. In particular, the first method S100 can be executed by a local or remote computer system that communicates with one or more optical inspection stations to collect images of the assembled units almost in real time, is interfaced with a local or remote database to obtain stored images, and / or hosts a user interface (e.g., at the user's smartphone, tablet, or desktop computer) to provide images and related data to the user and receive image selections and other inputs from the user. The optical inspection stations (described below) can be inserted into the assembly line at different assembly stages and immediately used to capture images of the units passing through the assembly line. The optical inspection stations can upload these images to a (local or remote) database, for example, in real time or asynchronously, with a timestamp of when each image was captured and an identifier of the optical inspection station that captured each image (e.g., "image metadata"). Then, the system can execute blocks of the first method S100 locally at a computer system directly connected to the assembly line, within a local application or web browser executed on a mobile computing device logged into the assembly line, or remotely at a remote server to automatically identify the optical inspection stations inserted into the assembly line, automatically identify the order of the optical inspection stations along the assembly line, and automatically determine the positions of the individual units along the assembly line at a particular (e.g., current) moment based on the visual data contained in the images and image metadata received from the optical inspection stations. Then, the system can automatically configure a virtual representation of the assembly line, including the relative positions of the optical inspection stations and the relative positions of the units along the assembly line, as shown in FIG.*2. Thus, the system can execute blocks of the first method S100 to automatically configure the optical inspection stations along the assembly line and generate a virtual representation of the state of the units within the assembly line for presentation to the user substantially in real time as the units are being assembled.

[0282] The system can execute the first method S100 to collect, process, and manipulate images of test components (hereinafter referred to as "units") during product development, for example, during prototype building, engineering verification testing (EVT), design verification testing (DVT), and / or production verification testing (PVT). The system collects, processes, and manipulates images of the units captured by one or more optical inspection stations during a prototype building event (or "build") over hours, days, or weeks of assembling and testing dozens, hundreds, or thousands of units. The system can also be implemented within a batch or mass production assembly line for in-process quality control, early defect detection, etc. during a production run. The system can also be integrated into a manual pass-through assembly line or a conveyor belt assembly line.

[0283] In addition, the system can be implemented across distributed assembly lines, such as assembly lines physically located in the same place or in different buildings installed on a single campus, on different campuses of the same company or different companies, and / or on assembly lines in different cities, regions, countries, or continents. For example, multiple sets of optical inspection stations can be installed within each of a plurality of discrete and remotely located assembly lines of a product or sub-assembly of a product, and the system can aggregate the images captured and uploaded by the optical inspection stations into a single distributed assembly line of the product or sub-assembly. Similarly, the system can be implemented during the production, testing, and / or verification of individual components, sub-assemblies, main assemblies, etc. in any sub-component or main assembly layer of raw material handling facilities, injection molding facilities, casting and machining facilities, assembly facilities, inspection facilities, testing and reliability testing facilities, validation or failure analysis facilities, packaging facilities, shipping facilities, field use facilities, field return facilities, and / or field return failure analysis, etc. Systems for integration with assembly lines are described herein. However, the system can be integrated with any one or more of the manufacturing, assembly, testing, validation, and / or other production processes of individual components, sub-assemblies, main assemblies (hereinafter referred to as "units"), etc.

[0284] 1.2 Optical Inspection Station

[0285] The system includes one or more optical inspection stations. Each optical inspection station can include: an imaging platform for receiving parts or components; a visible light camera (e.g., an RGB CMOS or black and white CCD camera) that captures an image (e.g., a digital photographic color image) of a unit placed on the imaging platform; and a data bus for unloading the image to, for example, a local or remote database. The optical inspection station can additionally or optionally include multiple visible light cameras, one or more infrared cameras, laser depth sensors, etc.

[0286] In one implementation, the optical inspection station further includes a depth camera configured to output a depth image, such as an infrared depth camera. In this implementation, the optical inspection station can trigger the visible light camera and the depth camera to respectively capture a color image and a depth image of each unit placed on the imaging platform. Optionally, the optical inspection station can include optical fiducial points arranged on and / or near the imaging platform. In this implementation, the optical inspection station (or a local or remote computer system interfaced with a remote database) can implement machine vision techniques to identify these fiducial points in the color image captured by the visible light camera, and convert the dimensions, geometry (e.g., distortion from a known geometry), and / or position of these fiducial points in the color image into a depth map, a three-dimensional color image, or a three-dimensional measurement space of the color image (described below).

[0287] The system is herein described as including one or more optical inspection stations and generating a virtual representation of an assembly line that includes one or more optical inspection stations. However, the system may additionally or alternatively include any other type of sensor-bearing station, such as an oscilloscope station that includes an NC control probe, a weighing station that includes a scale, a surface profiling station that includes an NC-controlled surface profiler, or a station that includes any other optical, acoustic, thermal, or other type of contact or non-contact sensor.

[0288] 1.3 Automatic Configuration

[0289] After a set of optical inspection stations are inserted into an assembly line, the optical inspection stations may capture color images of the units passing through the optical inspection stations and upload the color images to a local or remote database. After receiving an image from a deployed optical inspection station, the system may: implement optical character recognition technology or other machine vision technologies to identify and read the serial number, barcode, quick response (“QR”) code, or other visual identifier of the unit within the image; generate an alphanumeric label representing the serial number, barcode, QR code, or other visual identifier; and then add the alphanumeric label to the metadata received with the image. Thus, the system may receive images of various units in blocks S111, S112, and S113 and then read the identification information of these units from these images in blocks S121, S122, and S123. (Optionally, each optical inspection station may include an RFID reader, an NFC reader, or other optical or radio reader that locally reads the serial number from a unit placed on its imaging platform, and the optical inspection station may add the serial number read from the unit to the metadata of the image of the assembled unit.)

[0290] In block S130, the system may then process the unit serial number, optical inspection station identifier (e.g., serial number), and timestamp (i.e., the time when a unit with a known serial number enters an optical inspection station with a known identifier) included in the metadata of the images received from the optical inspection stations to determine the order of the optical inspection stations along the assembly line, as Figure 1As shown. In one implementation, when receiving an image from an optical inspection station, the system: stores (buckets) a set of images containing tags with specific unit serial numbers; extracts the optical inspection station serial number tag and the timestamp from the metadata in this set of images; and sorts these optical inspection station serial numbers (from the first to the last in the assembly line) according to their corresponding timestamps (from the oldest to the newest). In particular, the unit progresses through the assembly over time and is sequentially imaged by the optical inspection stations along the assembly line, and the system can convert the unit serial number, the optical inspection station serial number, and the timestamp stored together with the images received from these optical inspection stations into an identification of a set of optical inspection stations corresponding to an assembly line and a confirmation of the order of the optical inspection stations along this assembly line. The system can repeat this process for other unit serial numbers, such as for each serial number of the unit entering the first optical inspection station in this sorted group of optical inspection stations, in order to confirm the determined order of the optical inspection stations along the assembly line and automatically detect reconfigurations of the optical inspection stations on the assembly line (e.g., in real time).

[0291] In this implementation, the system can also pass these optical inspection station serial numbers into a name mapping system (such as DNS) to obtain optical inspection station-specific information, such as the brand, model, last user-entered name, configuration (such as imaging platform size, optical resolution, magnification capacity), owner or lessee, etc. for each optical inspection station. The system can similarly pass the unit serial numbers into a name mapping system or other database to obtain unit-specific data, such as the specified construction, configuration, bill of materials, special assembly instructions, measurements, photos, notes, etc.

[0292] In block S150, the system can then generate a virtual representation of the sorted optical inspection stations along the assembly line, as Figure 2 shown. The system can label the virtual representation of the optical inspection stations with the brand, model, name, configuration, serial number, etc. obtained from a remote database or based on the name or description entered by the user. Then, the system can upload the virtual representation of the assembly line to a local or remote computer system (such as a smartphone, tablet, desktop computer) for user access. The system can also receive images from optical inspection stations across multiple different assembly lines and can substantially implement the foregoing methods and techniques in real time to store images of units on different assembly lines, identify multiple assembly lines and the order of optical inspection stations in each assembly line, generate a unique virtual representation of each assembly line represented by the images, and assign these virtual assembly line representations to their corresponding owners.

[0293] The system can also repeat the foregoing methods and techniques throughout the operation of the assembly line in order to detect the insertion of additional optical inspection stations into the assembly line, detect the removal of optical inspection stations from the assembly line, and / or detect the rearrangement of optical inspection stations within the assembly line, and automatically update the virtual representation of the assembly line accordingly.

[0294] 1.4 Assembly Line Status

[0295] In block S140, the system can identify the current position of a unit within the assembly line based on the optical inspection station serial number tag stored together with the last image received from the assembly line that contains the unit serial number of the assembly unit. For example, for a unit within the assembly line, if the last image received from the second optical inspection station contains the unit serial number tag of a specific unit (i.e., the optical inspection station serial number tag of the first optical inspection station), the system can determine that the specific unit is between the first optical inspection station and the second optical inspection station along the assembly line. In this example, if the last image containing the assembly unit serial number tag of the specific unit is recently received from the second optical inspection station through another image of another unit that has not yet been received from the second optical inspection station, the system can determine that the specific unit is at the second optical inspection station along the assembly line. Additionally, in this example, if the last image containing the assembly unit serial number tag of the specific unit is received from the last known optical inspection station on the assembly line, the system can determine that the assembly of the specific unit has been completed.

[0296] The system can repeat the foregoing process for the unit serial numbers of other units identified in the images received from the optical inspection stations inserted along the assembly line. Then, the system can fill the virtual representation of the above assembly line with a heat map of the current unit positions, as Figure 1 and 2 shown. In block S150, when each new image is received from an optical inspection station on the assembly line and thus the new position of a specific unit along the assembly line is determined, the system can update the virtual representation of the assembly line to reflect the newly determined position of the specific unit. The system can also cause the marker representation of the specific unit within the virtual representation of the assembly line to bounce or otherwise be animated to visually indicate to the user that the specific unit has moved.

[0297] The system can implement similar methods and techniques to generate a heat map or other virtual representation of the assembly line at a specific previous time based on the last image of the unit received from the optical inspection stations along the assembly line before a specific previous time selected by the user. Thus, as the user scrolls through the entire time history of the assembly line, the system can recalculate the assembly line state and unit positions at previous times and display virtual representations of these assembly line states substantially in real time for the user. The system can also filter the images received from the optical inspection stations based on a user selection of a subset of the units on the assembly line, for example, according to configuration, setting, date or time, inspection time, etc.; then in block S150, the system can calculate the assembly line state for the subset of units from the filtered images and display a virtual representation of the assembly line state.

[0298] However, the system can perform the blocks of the first method in any other way to convert the images received from the optical inspection stations into the configuration of the optical inspection stations along the assembly line and determine the state of the units along the assembly line.

[0299] 1.5 Defect Detection

[0300] In one variation, the system implements machine vision techniques to detect manufacturing defects along the assembly line and uses the location, type, and / or frequency of the manufacturing defects detected in the units passing through the assembly line to enhance the virtual representation of the assembly line. For example, the system can implement the methods and techniques described below to analyze images of the units to detect features (e.g., part dimensions, absolute or relative part positions) that fall outside the dimensions and tolerances specified for the features. In another example, the system can implement template matching techniques to detect scratches, dents, and other aesthetic defects on the units in the images received from the optical inspection stations along the assembly line.

[0301] In this variation, when a defect on a unit is detected in the earliest image of the assembly unit, the system can mark the unit serial number corresponding to the image in which the defect is detected, and then insert the defect mark into the virtual representation of the assembly line at the specific optical inspection station where the image was captured. Thus, the system can visually indicate to the user, via the virtual representation of the assembly line, that the defect on the assembly unit appears between a specific optical inspection station in the assembly line and a second optical inspection station immediately preceding the specific optical inspection station. Additionally, if the system detects defects shown in multiple images captured at a specific optical inspection station, the system can identify the same type of defect (e.g., similar scratches in the same area on the housing spanning multiple units), and merge the counters of defects of the same defect type into the virtual representation of the assembly line. The system can also visually represent, for example in the form of a heat map, the frequency, type, and / or location of the detected defects of a batch of units passing through one or more optical inspection stations. For example, the system can generate or access a virtual representation of a "nominal" or "typical" unit, calculate a heat map of the visual representation of the total defects detected in similar units passing through a single optical inspection station or through multiple optical inspection stations in the assembly line, and then present the heat map overlaid on the virtual representation of the nominal unit within the user interface.

[0302] However, the system can implement any other method or technique to identify defects in the units shown in the images captured by the optical inspection stations within the assembly line and indicate the earliest detected presence of the defect on the assembly unit in the virtual representation of the assembly line.

[0303] 2. Window Mapping

[0304] As Figure 3 shown, a second method S200 for reproducing images of assembly units along an assembly line includes: in block S210, reproducing a first image of a first assembly unit captured at an optical inspection station within a user interface; in block S220, selecting a first feature within the first image corresponding to the first assembly unit as the origin of the first image; in response to receiving a zoom input, in block S212 reproducing an extended region of the first image within the user interface; and in block S230 storing the dimensions and position of the extended region relative to the origin of the first image. The second method S200 further includes: in response to advancing from the first image to a second image of a second assembly unit captured at the optical inspection station, in block S240 selecting a second feature within the second image corresponding to the first feature in the first image as the origin of the second image; in block S242 selecting an extended region of the second image corresponding to the dimensions and position of the extended region of the first image relative to the origin of the second image; and in block S250 reproducing the extended region of the second image within the user interface.

[0305] A variation of the second method S200 includes: in block S210, displaying a first image of a first assembly unit within a user interface, the form of the first image being recorded at an optical inspection station; in block S220, positioning a first virtual origin at a first feature on the first assembly unit represented in the first image; in response to a change in a view region of the first image at the user interface, in block S212, displaying a first sub-region of the first image within the user interface; and in block S230, recording the geometry and position of the first sub-region of the first image relative to the first virtual origin. The second method S200 further includes in response to receiving a command at the user interface to advance from the first image to a second image of a second assembly unit: in block S240, positioning a second virtual origin at a second feature on the second assembly unit represented in the second image, the second feature on the second assembly unit being similar to the first feature on the first assembly unit; in block S242, projecting the geometry and position of the first sub-region of the first image onto the second image according to the second virtual origin to define a second sub-region of the second image; and in block S250, displaying the second sub-region of the second image within the user interface.

[0306] As Figure 4 shown, another variation of the second method S200 includes: in block S210, displaying a first image of a first assembly unit within a user interface, the first image being recorded at an optical inspection station; in response to a change in a view window of the first image at the user interface, in block S212, displaying a first sub-region of the first image within the user interface; in block S230, recording the geometry and position of the first sub-region of the first image relative to a first feature represented in the first image; in block S240, identifying a second feature represented in a second image of a second assembly unit, the second feature being similar to the first feature; in block S242, projecting the geometry and position of the first sub-region of the first image onto the second image according to the second feature to define a second sub-region of the second image; and in response to receiving a command at the user interface to advance from the first image to the second image, in block S250, displaying the second sub-region of the second image within the user interface to replace the first image.

[0307] 2.1 Application

[0308] Blocks of the second method S200 may be locally executed by a computing device (such as a smartphone, a tablet computer, a desktop computer) communicating with a remote computer system to display images of a set of assembled units (or “units”) captured by an optical inspection station along an assembly line. During an image viewing session, the computing device may display images of the units of one or more assembly configurations across one or more configurations within a user interface reproduced on a display integrated into or connected to the computing device (e.g., within a native application or a web browser executed on a smartphone, a tablet computer, or a desktop computer).

[0309] Generally, when a user scrolls through an entire set of images of units previously captured by a particular optical inspection station, the computing device may execute blocks of the second method S200 to automatically apply the most recent zoom level and viewing position from one image reproduced in the user interface to the next image selected by the user, enabling the user to scroll through a sequence of images of multiple units recorded at the same point along the assembly line in the same expanded view. In particular, when a user scrolls from a first image to a second image within a set of images of similar units, the computing device (or a desktop or handheld computing device interfaced with the computing device) may execute blocks of the second method S200: to map a last observation window at the same assembly stage from the first image of the first unit to the second image of a similar second unit based on features within the first and second images; and to automatically reproduce an expanded view of the second image in the user interface (which corresponds to the last reproduced expanded view of the first image), such that the user may quickly and visually compare local differences between the first and second units relative to a common virtual origin.

[0310] In one example, a computing device can aggregate a set of images of PCB assemblies (including PCBs and components attached to the PCBs) captured by a particular optical inspection station during construction. When a user reviews a first image in the set during a first image review period, the computing device can: implement machine vision techniques to detect the upper left corner and upper horizontal edge of a first PCB shown in the first image; set the upper left corner of the first PCB as the virtual origin in the first image; align the X-axis of the virtual origin in the first image with the upper horizontal edge of the first PCB; reproduce a region of the first image in a user interface; store the X and Y coordinates of the upper left corner pixel, upper right corner pixel, lower left corner pixel, and lower right corner pixel of the first image reproduced in the user interface relative to the virtual origin for the last zoom level and viewing position of the first image. When the user then scrolls to a subsequent image in the set, the computing device can similarly: implement machine vision techniques to detect the upper left corner and upper horizontal edge of a second PCB shown in the second image; set the upper left corner of the second PCB as the virtual origin in the second image; align the X-axis of the virtual origin in the second image with the upper horizontal edge of the second PCB; and immediately reproduce a rectangular region of the second image delimited by four corner pixels at the corner pixel coordinates stored at the last viewing position of the first image. In this example, because the first image and the second image are (virtually) translationally aligned by a similar virtual origin and rotationally aligned by a similar X-axis reference feature, and because the relative position of the first region of the second image reproduced in the user interface - once the user scrolls to the second image - is substantially the same as the relative position of the last region of the first image reproduced in the user interface, the user can visually detect differences in component placement in the first PCB assembly shown in the first image and component placement in the second PCB assembly shown in the second image. When the user scrolls back from the second image to the first image or when the user scrolls from the second image to a third image in the set, the computing device can repeat the process. As Figure 5 shown, the computing device can also implement blocks of a second method S200 to simultaneously reproduce similar regions of images of, for example, similar cells in a grid layout or aligned and overlaid on top of each other in view windows.

[0311] Blocks of the second method S200, as performed by a "system", are described below. For example, blocks of the second method S200 can be implemented by a local computing device (such as a smartphone, tablet, or desktop computer) that executes a user interface. Optionally, blocks of the second method S200 can be remotely executed, for example, at a remote server connected to the local computing device via an interface to provide images to the user and receive image filter parameters, image selections, and other inputs from the user.

[0312] 2.2 Image

[0313] Block S210 of the second method S200 recites displaying a first image of a first assembly unit within a user interface, where the form of the first image is recorded at an optical inspection station. Typically, in block S210, the system acquires a first image of the first assembly unit and presents the first image to the user via the user interface; at the user interface, the user can then zoom into various regions of the first image and move the first image vertically and horizontally within a zoom window to visually remotely inspect regions of the first assembly unit represented in these regions of the first image.

[0314] 2.2.1 Homography Transformation

[0315] In one implementation, in block S210, the system obtains from a database a first digital photographic image previously recorded by the optical inspection station during assembly. The system then normalizes the first digital photographic image to generate the first image, which can then be presented to the user at the user interface. For example, the optical inspection station can include a digital photographic camera and a wide-angle lens coupled to the digital photographic camera; thus, the image recorded by the optical inspection station may exhibit perspective distortion. During the establishment of the optical inspection station, a reference object defining a reference surface, such as a 300 square millimeter white plane with black orthogonal grid lines at a known offset distance of 10 millimeters, can be placed within the optical inspection station, and the optical inspection station can record a "reference image" of the reference surface and upload the reference image to a remote database. The system can then: obtain the reference image; implement computer vision techniques to identify the distorted grid lines in the reference image; and then calculate a homography transformation that maps the distorted grid lines in the reference image to straight orthogonal grid lines. In this example, the system can also calculate a scalar coefficient that associates digital pixels with real dimensions (i.e., length values in real space) based on the known distances between the grid lines on the reference surface. Thus, the system can apply the homography transformation to the first digital photographic image to generate a "flattened" (or "undistorted") first image, and then display the first image (now free of perspective distortion) in the user interface for presentation to the user. As described below, the system can also extract the real dimensions of the features of the first assembly unit from the first image by summing multiple pixels in the first image across a feature and then multiplying this number of pixels by the scalar coefficient.

[0316] In the foregoing implementation, the system can transform all other digital photographic images recorded at the same optical inspection station during the assembly of a specific assembly type at a specific assembly stage according to the same homography transformation; the system can also apply the same scalar coefficient to the resulting flattened images. For example, when a new digital photographic image is received from an optical inspection station, the system can: immediately calculate the corresponding flattened image based on the homography transformation unique to that optical inspection station; and then store the original digital photographic image and the corresponding flattened image together in a database. As described below, the system can also generate a measurement space of the original digital photographic image, a compressed form (e.g., thumbnail) of the flattened image, a feature space of the flattened image, and / or other images, spaces, or layers related to the digital photographic image or the flattened image, and store this data together (e.g., in a single file associated with the corresponding assembly unit) in the database. Optionally, the system can store the digital photographic image in the database and then generate the corresponding flattened image in real time when a review of the corresponding assembly unit is requested at the user interface.

[0317] 2.2.2 Image Group

[0318] The system can define a set of related images by the assembly units represented in these images. For example, the optical inspection station can store a timestamp and an optical inspection station identifier in the metadata of the image; the system can also write the assembly type and the assembly stage to the image metadata based on the known positions of the optical inspection stations along the assembly line. The system can also implement computer vision techniques to read a serial number or other optical identifier from a region of the image representing the assembly unit or the fixture that locates the assembly unit with the optical inspection station, and can write this serial number of the other identifier to the image metadata. In addition, the system can determine the configuration of the assembly unit represented in the image based on the timestamp, serial number, and / or assembly stage, etc. of the assembly unit, and write this configuration to the image metadata. Similarly, the system can implement computer vision techniques (e.g., template matching, pattern matching, object recognition) to directly extract the assembly type and / or assembly status of the assembly unit represented in the image from the image. For the images stored in the database, the system can repeat this process asynchronously, and for the new images received from the deployed optical inspection stations, the system can repeat this process (near) in real time.

[0319] Then, the system can apply various filters to the metadata stored with these images to define a set of relevant images. For example, the system can automatically aggregate all the images recorded at an optical inspection station during an assembly cycle or "build" (e.g., EVT, DVT, or PVT) to define this set of images. Similarly, the system can select a set of images representing a set of assembly units of the same assembly type and in the same assembly state from a body of images of multiple assembly units recorded across a set of optical inspection stations and representing various assembly states. The system can also receive a set of filter parameters from the user, such as a time window, configuration, configuration, assembly stage, and / or other filters, as described below, and populate this set of images according to the filters. In addition, the system can sort the images in the set, for example, by timestamp or serial number, and display the images in this order as the user scrolls through the entire set of images within the user interface.

[0320] 2.3 View Window

[0321] Block S212 of the second method S200 recites displaying a first sub-region of the first image within the user interface in response to a change in the view window of the first image at the user interface. Generally, in block S212, the system receives an input at the user interface, interprets the input as a command to change the view window of the first image currently reproduced within the user interface, and updates the view window accordingly.

[0322] In one implementation, in block S210, the system initially displays the first image at its full height and width within the user interface. When a zoom input is received at the user interface, for example, via a scroll wheel, selecting a zoom level from a drop-down menu, or a zoom level slider, in block S212, the system redefines the view window to enclose a smaller region of the first image and reproduces the smaller region of the first image delimited by the view window at a higher resolution within the user interface. Then, the system can implement the methods and techniques described below to select a virtual origin within this new view window and define the geometric shape and position parameters of the view window relative to the virtual origin.

[0323] Once the first image is zoomed, the user can drag or move the first image vertically or horizontally relative to the view window. Then, the system can select a new virtual origin within the modified view window and / or redefine the geometric shape and position parameters of the modified view window relative to the current virtual origin. With each change in the zoom level and the position of the first image relative to the view window, the system can automatically implement the process of updating the region of the first image and the resolution of the region displayed in the user interface, reselecting the virtual origin of the first image (e.g., if the previous virtual origin is no longer within the view window), and automatically recalculating the geometric shape and position parameters of the view window relative to the current virtual origin.

[0324] Optionally, the system may: automatically update in real time the region of the first image displayed in the user interface and the resolution of that region in response to changes in the zoom level and position of the first image within the view window; and in response to manually entering a command through the user interface to store the current view window and populate the view window across the other images in the group, select the virtual origin of the first image and selectively recalculate the geometric shape and position parameters of the view window relative to the virtual origin.

[0325] However, the system may implement any other method or technique to update the region and resolution of the first image reproduced within the user interface, and automatically or selectively trigger the selection of the virtual origin in the first image in block S220 and the recording of the view window parameters in block S230.

[0326] 2.4 First Image: Origin Selection

[0327] Block S220 of the second method S200 recites positioning a first virtual origin at a first feature on a first assembly unit represented in the first image in block S220. Generally, in block S220, the system positions the virtual origin within the first image relative to a distinguishable feature (e.g., on the distinguishable feature) within the first image; then, in block S230, the system may define a view window of the current region of the first image reproduced in the user interface relative to that virtual origin. Because the system positions the virtual origin at a distinguishable feature within the first image, the system may implement computer vision techniques to identify similar (e.g., like) distinguishable features in the other images in the group, and similarly position the virtual origin relative to those similar features (e.g., on those similar features). By reproducing the images in the group within the user interface with their virtual origins at the same location and at the same scale and resolution as the first image, the system may maintain a set of view windows for the first image across the other images in the group, enabling the user to view (e.g., scroll through) regions of images of different assembly units positioned relative to a common feature represented in those images.

[0328] In particular, over time, the positions of like parts, components, and subassemblies may not be located in the same positions and orientations relative to the global assembly unit and other parts, components, and subassemblies within the assembly unit along a group of assembly units assembled along the same assembly line. Additionally, fixtures configured to constrain the assembly unit within an optical inspection station may exhibit non-zero positional tolerances such that the assembly unit captured in an image sequence may shift significantly from one image to the next. To maintain a view window from a first image of a first assembly unit to a second image of a second assembly unit, the system may define a first virtual coordinate system within the first image, e.g., including a virtual origin and virtual axes, and define the view window relative to the first virtual coordinate system. The system may then, e.g., define a similar second virtual coordinate system within the second image relative to similar features of the first and second assembly units captured in the two images, and project the view window onto the second image based on the second virtual coordinate system. By defining a reference coordinate system across a group of images relative to or based on similar features within the assembly units represented in the image group, the system may display these similar features in the same position within the user interface, enabling a user to quickly and visually distinguish differences in the relative positions of other features within these assembly units relative to these similar features as the user indexes through the entire group of images within the user interface.

[0329] 2.4.1 Manual Origin Selection

[0330] In one implementation, in block S220, the system may select a global feature within the first image and define a global virtual origin across the entire region of the first image. In one example, the system: reproduces the first image using the user interface (e.g., the entire image of the first image or an extended sub-region of the first image); overlays a curve on the first image aligned with the features of the first assembly unit shown in the first image; receives from the user a selection of one or more points, the intersection of two curves, an entire curve, etc.; and then converts the user's selection into the virtual origin of the first image and the axes of the virtual coordinate system.

[0331] When viewing a first image within a user interface, a user may initiate a new view window specification, for propagation across the set of images, for example, by selecting a label or other input area within the user interface. Before presenting the first image to the user or once the user initiates a new view window specification, the system may: implement edge detection and / or other computer vision techniques to identify edges of real features on a first assembly unit represented in the first image; generate a feature space placed on the first image; and fill the feature space with colored (e.g., yellow, green) vectors that align with and represent the edges of the corresponding real features shown within the first image. In this implementation, the system may generate line or curve vectors representing the edges of features of the first assembly unit shown in the first image; the system may also interpolate regions of features delimited by three or more straight lines or one or more curves. Additionally, the system may identify points within the first image, such as the ends of lines and curves, intersection points of lines and curves, corners of regions, and / or centroids of regions (e.g., centers of square, rectangular, and circular regions). Once the user initiates a new view window specification, the system may activate the feature space, for example, by reproducing the feature space placed on the first image or by highlighting vectors within the feature space near the cursor as the user manipulates the cursor within the user interface. The system may then store the user's selection of one or more vectors from the feature space and transform these vectors into a virtual origin of the first image, as described below.

[0332] Similarly, when viewing the first image within the user interface (and after initiating a new view window specification), the user can select a pixel within the current view window. The system can then select the point, line, or curve in the first-image-specific feature space that is closest to that pixel. Optionally, upon receiving the selection of the pixel, the system can implement the above methods and techniques to scan the area of the image surrounding the selected pixel at the edge, generate a set of vectors representing these nearby features, and then select the vector that is closest to the selected pixel. The system can also reproduce the vector on the first image to indicate the specific feature. Optionally, the system can prompt the user to select multiple (e.g., three) pixels around a feature (e.g., a point, corner, or center of a region) within the first image or along a feature (e.g., along an edge) within the first image, and then implement similar methods and techniques to identify the single feature (e.g., the edge, corner, or center of the region) that is closest to these multiple pixels. The system can then store this feature selected by the user and accordingly implement the methods and techniques described below to define the virtual origin in the first image. Thus, the system can: identify a set of edge features on a first assembly unit within a first sub-region of the first image; receive a selection of a pixel within the first sub-region of the first image; select a first feature from the set of edge features that is closest to the pixel; and locate the virtual origin on the first feature. However, the system can interface with the user in any other way through the user interface to receive a manual selection of a reference feature within the first image or receive a selection of a reference origin within the first image.

[0333] In this implementation, if a user directly selects a point feature within the first image through the feature space or indirectly by selecting pixels, such as the intersection point of two curves (e.g., a corner), the end of a line, or the center of a surface, the system can set a virtual origin in the first image at that point feature. The system can also define axes for the first image. For example, for a point feature at the end of a line or curve that lies within the feature space, the system can define a virtual axis that intersects the virtual origin and is tangent to the line or curve. Similarly, for a point feature at the intersection of two curves (e.g., a corner) that lies within the feature space, the system can define: a first virtual axis that intersects the virtual origin and is tangent to the first curve; and a second virtual axis that intersects the virtual origin and is tangent to the second curve. Thus, in this example, when positioning the first virtual origin in the first image, the system can also position the first virtual origin of the first coordinate system on the first feature in the first image, and the system can also align the first axis of the first coordinate system with the first feature. However, for a point feature that lies within a surface, the system can identify the closest line (e.g., an edge) feature that delimits the surface and align the virtual axis with that line feature; optionally, the system can detect an edge feature that delimits the global boundary (e.g., the bottom edge, the left side) of the first assembly unit and align the virtual axis with that edge feature. The virtual axis and the virtual origin can thus cooperate to define the virtual coordinate system of the first image. However, the system can implement any other method or technique to place the virtual origin and the virtual axis in the first image.

[0334] 2.4.2 Automatic Global Origin Selection

[0335] Optionally, in block S220, the system can automatically detect a reference feature on the first assembly unit shown in the first image and then define the virtual origin of the first image relative to that reference feature.

[0336] In one implementation, the system automatically calculates the default virtual origin of the images in the set. For example, the system can: implement machine vision techniques (e.g., edge detection, template matching) to detect the maximum perimeter of the first unit shown in the first image; identify the upper left corner of the maximum perimeter of the first assembly unit; define the virtual origin of the first image at this upper left corner; implement machine vision techniques to detect the straight edge closest to the virtual origin and / or intersecting the virtual origin; and then align the virtual axis of the virtual coordinate system with that closest straight edge.

[0337] Similarly, the system can define a virtual origin at a feature on a fixture that confines a first assembly unit within an optical inspection station, as represented in a first image. For example, the system can: implement template matching or other object recognition techniques to detect fiducial points, such as quick response codes, engraved true coordinate systems, a set of three pins or polished steel balls, corners of a fixture plate, or other known optical markers on the fixture plate, in a region of the first image outside the perimeter of the first assembly unit (e.g., the fixture); and then place the virtual origin at this fiducial point.

[0338] In another example, the system can: implement machine vision techniques to identify the upper left corner of the perimeter of the topmost linear component shown in the first image (i.e., the component closest to the camera in the station where the first image is captured); define the virtual origin of the first image at this upper left corner; implement machine vision techniques to find the closest straight edge on the topmost linear component shown in the first image; and then align the virtual axes of the virtual coordinate system of the first image with this straight edge, as Figure 3 shown.

[0339] 2.4.3 Automatic Origin Selection within Zoom Window

[0340] As Figure 4 shown, the system can implement similar methods and techniques to automatically select features within a first image delimited by a view window and define a virtual origin relative to such features, for example, in response to a change in the view region of the first image at a user interface. In particular, in this implementation, the system can implement similar methods and techniques to select local features within a region of the first image reproduced within the user interface and define a local virtual origin relative to such local features. For example, in response to each change in the zoom level and viewing position of the first image during an image viewing session, the system can recalculate the virtual origin and virtual axes (e.g., the orientation of the virtual coordinate system) relative to features of the first assembly unit shown in an extended region of the first image currently reproduced within the user interface.

[0341] The system can implement the above methods and techniques to: detect edges generally within the first image or within a region of the first image currently delimited by a view window; identify a set of discrete surfaces on a first assembly unit delimited by these edges; select a particular discrete surface presenting the largest dimension, such as the largest area or the largest length within the set of discrete surfaces; select a feature delimiting a portion of the particular discrete surface (e.g., the longest edge of the particular discrete surface represented in the region of the first image reproduced in the user interface); position a virtual origin on the feature within the first image, such as at the uppermost and / or leftmost end of the feature displayed in the view window; and then align the virtual axes parallel or tangential to the feature. Optionally, the system can: calculate the centroid of the particular surface; define the virtual origin of the first image at the centroid; detect edge features delimiting the particular curved surface and align the virtual axes with this edge feature. Thus, the system can automatically place the virtual origin and virtual axes in the first image based on the largest surface within the image region delimited by the current view window. As described above, the system can implement similar methods and techniques to detect surfaces represented within the first image that are delimited by the current view window and positioned closest to the camera that captured the first image.

[0342] Similarly, the system can: detect a set of edge features within a region of the first image delimited by the current view window; detect the intersection points (i.e., "corners") of these edge features; position the virtual origin at the uppermost left corner detected within the region of the first image; and align the virtual axes with the edge features intersecting this corner. Similarly, the system can position the virtual origin at the corner closest to the center of the current view window.

[0343] Thus, the system can detect edge features delimiting the boundaries of parts within the first assembly unit, detect the corners of these parts, and place the virtual origin at a corner of one of these parts represented within the region of the image currently reproduced within the user interface, one of the parts being, for example, the largest part, the part closest to the upper left corner of the view window, or the part at the highest height within the region of the first assembly unit represented within the region of the image.

[0344] Optionally, the system may: implement computer vision techniques, such as object recognition or template matching, to identify different parts or part types within a sector of a first assembly unit represented in a region of a first image delimited by a current view window; identify a common reference part or common reference part type; and then define a virtual origin of the first image relative to the common reference part or common reference part type. For example, the system may identify: an edge of a PCB; a radial center of a camera lens; a node of an antenna; and / or a fastener head or fastener hole within a region of the first image. The system may then implement the above methods and techniques and a predefined hierarchical structure of common reference parts or part types to select the highest-level part or part type, select features that delimit or bound the part or part type, and define a virtual origin and virtual axes based on the feature. For example, the part type hierarchical structure may prioritize fixtures, then assembly units (such as enclosures), parts (such as PCBs), sub-parts (such as integrated circuits or other chips mounted to a PCB), etc. In this example, the system may implement computing device techniques to identify features representative of these component types within a region of a first image delimited by a view window, and then position a virtual coordinate system on the highest-level part type identified within that region of the first image according to the part type hierarchical structure.

[0345] In the foregoing implementation, the system may evolve and modify the part or part type hierarchical structure over time. For example, the system may implement machine learning techniques to: track and characterize manual feature selections as described above; detect patterns in these manual feature selections; develop models for detecting similar features in images; and improve the part or part type hierarchical structure to automatically select representative features within regions of images of assembly units shown within an assembly unit over time.

[0346] However, in block S220, the system may implement any other method or technique to automatically position a virtual origin and / or virtual axes within a first image relative to one or more features represented in a region of the first image delimited by a current view window. Additionally, the system may, for example, switch between setting a global virtual origin and setting a local virtual origin for a first image currently viewed by a user based on a state of virtual radio buttons reproduced within a user interface or when the user zooms in from a lowest zoom level to the first image.

[0347] 2.5 View Window Specification: Last Viewed Area Parameters

[0348] Block S230 of the second method S200 describes recording the geometry and position of a first sub-region of the first image relative to a first virtual origin. Generally, in block S230, the system records parameters characterizing the current view window on the first image relative to the virtual origin and / or virtual axes defined for the first image. In particular, the system may record the geometry and position of the current view window relative to the virtual origin of the first image - defining the region of the first image currently reproduced within the user interface. The system may then store this data in a new view window specification.

[0349] In one implementation, as the user zooms in and out of the first image and re-positions the first image vertically and horizontally within the user interface, in block S230, the system may store parameters defining the last region of the first image reproduced within the user interface. For example, the system may store: the pixel width and pixel height of a rectangular region of the first image reproduced within the user interface; the horizontal and vertical pixel offsets of the top-left corner between the rectangular region and the virtual origin of the first image; and the angular offset between an edge of the rectangular region and the virtual axis of the first image in the new view window specification. In another example, the system may store the pixel coordinates of each corner of the rectangular region of the first image reproduced within the user interface relative to the virtual coordinate system defined by the virtual origin and virtual axes in the first image. In block S242, the system may implement these parameters to project the view window of the first image onto other images in the set.

[0350] The system may also write the zoom level at which the first image is currently being viewed and / or the ratio of the true size to the pixel size of the current zoom level. In block S242, the system may implement this data to set the zoom level of other images in the set, or scale these images to match the zoom level or ratio of the first image.

[0351] The system may also store the position of the feature selected to define the virtual origin and virtual axes in the first image and / or the position of a narrow feature window containing the feature in the first image, e.g., relative to the top-left most corner of the first image, relative to a reference point on a fixture shown in the first image or other reference benchmark, or relative to another global origin of the first image. Similarly, the system may characterize the feature, e.g., by classifying the feature as a corner, line, curve, arc, or surface and calculating the length, radius, or area of the feature (e.g., in pixel-based units or real units). For example, the system may store these parameters in a new view window specification. The system may then implement these parameters in block S240 to identify similar features in other images and locate comparable virtual origins and virtual axes in these other images.

[0352] However, the system can store any other set of values representing the size and location of the region of the first image that was last reproduced in the user interface.

[0353] In addition, in block S230, the system can implement similar methods and techniques to position the view window of the first image directly relative to a feature or a set of features within the first image, e.g., not relative to an origin or coordinate system located on the feature.

[0354] 2.6 Second Image: Origin Selection

[0355] Block S240 of the second method S200 recites positioning a second virtual origin relative to a second feature represented in a second image of a second assembly unit, where the second feature is similar to the first feature. (Block S240 can similarly recite positioning the second virtual origin at a second feature on the second assembly unit represented in the second image in response to receiving a command at the user interface to advance from the first image to the second image of the second assembly unit, where the second feature on the second assembly unit is similar to the first feature on the first assembly unit.) Generally, in block S240, the system automatically identifies a second feature in the second image of the second assembly unit that is similar (e.g., similar in position and geometry) to the first feature that was selected in the first image to define the first virtual origin and / or the first virtual axis in the first image. Once the second feature in the second image is identified in block S240, the system can define a second virtual origin and a second virtual axis for the second image based on that second feature. In particular, in block S240, when the user scrolls from the first image to the second image, the system implements the above methods and techniques to automatically identify a second reference feature in the second image that is substantially the same as the first reference feature selected in the first image, and automatically define the virtual origin in the second image based on that second reference feature.

[0356] In one example, each time the user adjusts the view window of the first image or saves a new view window specification, the system performs block S240 on all other images in the set, the next five and previous five images in the set of images, or the next and previous images in the set. Optionally, once the user advances (e.g., scrolls forward or backward, moves forward or backward on an option) from the first image to the second image, the system can perform block S240 to identify a similar feature in the second image.

[0357] 2.6.1 Bounded Scanning Area

[0358] In one implementation, the system: projects a boundary that encloses and is offset from a first virtual origin of a first image onto a second image; identifies a set of edge features on a second assembly unit represented within a region of the second image that is enclosed by the boundary; identifies a second feature within the set of edge features, the second feature exhibiting a second geometry that is approximately similar to a first geometry of a first feature from the first image; and locates a second virtual origin on or relative to the second feature according to parameters implemented by the system when the first origin in the first image is located. For example, to detect the second feature in the second image, the system may: detect a global origin in the second image; project a feature window (described above) stored in a new view window specification onto the second window according to the global origin of the second image; and scan a region of the second image delimited by the feature window to find a feature that exhibits a geometry similar to the geometry of the first feature of the first image.

[0359] Optionally, the system may scan the entire second image to find a second feature that is similar to the first feature selected in the first image.

[0360] In block 240, the system may also detect and compare other reference features (e.g., the lower right corner of a PCB shown in the image) in the first image and the second image, determine whether the scale of the second image is globally or locally different from the first image based on the relative positions of the reference features in the first image and the second image, and then scale the second image if necessary so that a first view of the second image matches a last view of the first image previously reproduced in the user interface.

[0361] Optionally, before providing the images in the set to the user interface for viewing, the system may identify and locate multiple similar features across multiple images of multiple assembly units, e.g., in a feature space specific to each image, e.g., to speed up the process of projecting a view region of one image onto another image based on the positions of the similar features as the user scrolls through all the images in the set.

[0362] 2.6.2 Similar Feature Detection

[0363] In one implementation, the system: identifies a set of features represented in a second image (e.g., spanning the entire second image or within a region of the second image delimited by a feature window); selects a second feature within the set of features, the second feature exhibiting dimensional and geometric features that approximate those of a first feature; and then locates a second virtual origin on the second feature within the second image in block S240. For example, the system may implement pattern matching techniques to match the second feature in the second image with the first feature in the first image. Similarly, in the foregoing implementation, where in block S220 the system detects a first part within a first assembly unit and locates a first virtual origin on the first part, the system may: implement computer vision techniques (e.g., template matching, object recognition) to identify a set of parts within a second assembly represented in the second image; select a second part within the set of parts that is similar to the first part, the second part representing, for example, the geometry and location relative to other parts within the second assembly unit that is similar to the geometry and location of the first part within the first assembly unit relative to other parts within the first assembly unit; and then locate a second virtual origin in block S240 on a second feature of the second part in the second image, such as a corner of the second part that is similar to a corner of the first part, where the first virtual origin was located on that corner of the first part in block S220.

[0364] In the foregoing example, the system may: implement methods and techniques to generate a second feature space for the second image; compute an optimal alignment between the second feature space of the second image and the first feature space of the first image; and select from the second feature space a second feature that falls near a first feature within the aligned first feature space and exhibits a geometry (e.g., length and shape) similar to that of the first feature.

[0365] However, the system may implement any other methods and techniques to automatically detect a second feature (e.g., a point, line, curve, or area feature) in the second image that is similar to the first feature in the first image (i.e., the first feature used by the system to locate the first virtual origin and first virtual axis in the first image). Then, in block S240, the system may repeat the process of locating the first virtual origin and first virtual axis with respect to the first feature in the first image in block S220 to locate a second virtual origin and second virtual axis (e.g., a second coordinate system) with respect to the second feature in the second image.

[0366] 2.7 View Window Projection and Display

[0367] Block S242 describes projecting the geometry and position of a first sub-region of a first image onto a second image according to a second virtual origin to define a second sub-region of the second image; and block S250 describes displaying the second sub-region of the second image within a user interface. Generally, in block S242, the system selects a region of the second image to be initially reproduced within the user interface based on a set of parameters that define the last region of the first image previously reproduced within the user interface. In particular, in block S242, the system projects the last view window of the first image onto the second image based on the second virtual origin of the second image to define a similar view window of the second image. Thus, in block S250, when the system replaces the region of the first image delimited by the view window with the second image delimited by the similar view window within the user interface, a second feature in the second image (which is similar to a first feature in the first image) is reproduced within the user interface at the same position and in the same orientation as the immediately preceding first feature. Thus, in blocks S220 and S240, the system can locate coordinate systems in the first and second images respectively based on similar reference features common to the first and second images, and project the view window from the first image onto the second image in block S242 such that when, in block S250, the sub-region of the second image is reproduced within the user interface to replace the sub-region of the first image, the reference feature in the second image is aligned with its similar reference feature in the first image after translation and rotation. For example, when the user scrolls from the first image to the second image, the system can reproduce the sub-region of the second image within the user interface substantially in real time in block S250. When the user scrolls back from the second image in this set of images to the first image or scrolls from the second image to a third image of a third assembly unit, the system can repeat blocks S240, S242, and S250 at a later time.

[0368] In one implementation, in block S242, the system: defines the geometry of the view window of the second image according to the zoom level stored in the new view window specification; vertically offsets the origin of the view window from the second virtual origin in the second image according to the vertical offset stored in the new view window specification; horizontally offsets the origin of the view window from the second virtual origin in the second image according to the horizontal offset stored in the new view window specification; rotates the view window relative to the second coordinate system in the second image according to the angular offset stored in the new view window specification; and defines the region of the second image delimited by the new view window as the second sub-region. Thus, when recording the first and second images, the system can translate and rotate the view window in the second image to align with the view window in the first image to compensate for the local position and orientation changes of the parts within the first and second assembly units and the global position and orientation changes of the first and second assembly units within the optical inspection station. Then, in block S250, the system can display the second sub-region of the second image within the user interface to replace the first sub-region of the first image.

[0369] Optionally, in block S242, the system can implement similar methods and techniques to position the view window from the first image onto the second image relative to a second feature or set of features within the second image. However, in block S242, the system can implement any other method or technique to project the view window of the first image onto the second image and display the corresponding region of the second image within the user interface, with the reference features of the second image aligned with similar reference features of the first image previously displayed in the user interface.

[0370] 2.8 View Window Propagation

[0371] In a variation, the system provides indicators of the second feature, second virtual origin, and / or second virtual axis automatically selected for the second image in blocks S240 and S242, as well as a prompt to confirm these selections with the user via the user interface. After receiving confirmation of these selections from the user, the system can repeat the processes implemented in blocks S2,40 and S242 for all other images in the group.

[0372] In one implementation, once the system automatically selects the geometry and position of the second sub-region of the second image in block S242 and reproduces the second sub-region of the second image within the user interface in block S250, the system provides a prompt to the user via the user interface to confirm the geometry and position of the second sub-region before performing these processes on the other images in the set. If the user indicates via the user interface that the geometry and position are incorrect - for example, angularly offset, vertically or horizontally shifted, or incorrectly scaled with respect to the first sub-region of the previously displayed first image - the system can: repeat blocks S240 and S242 for the second image to recalculate the geometry and position of the second sub-region of the second image; display the modified second sub-region of the second image in the user interface; and similarly prompt the user to confirm the geometry and position of the modified second sub-region. When receiving an indication that the second sub-region of the second image is incorrect, the system can also prompt the user to select alternative second features in the second image and / or indicate a preferred origin, axis, and / or coordinate system of the second image, for example, by selecting alternative pixels within the second image as described above or selecting alternative features from a second feature space placed on the second image. Then, the system can modify the second sub-region of the second image based on these additional selections input by the user and update the process for propagating the view window across the set of images, for example, stored in a new view window specification. For example, the system can implement machine learning techniques to improve the process or model for automatically selecting similar features, locating the virtual origin, and determining the direction of the virtual axis across a set of related images based on the feedback provided by the user.

[0373] However, in response to receiving confirmation of the projected geometry and projected position of the second sub-region of the second image, the system can: obtain a set of images of other assembly units from the database as in block S210; locate the virtual origin in each image in the set of images as in block S240; and project the geometry and position of the first sub-region of the first image onto each image in the set of images in block S242 to define a set of sub-regions of the set of images. In particular, once the user confirms that the system correctly defines the second sub-region in the second image, the system can propagate, across all images in the set, the last view window of the first image defined, for example, in the new view window specification. Then, the system can pan within the entire set of sub-regions displayed within the user interface in response to a scroll input at the user interface, as described above in block S250. However, the system can implement any other method or technique to prompt, collect, and respond to user feedback related to the automatic selection of the second sub-region of the second image.

[0374] As described above, once the user confirms the second sub-region of the second image, for example before the user scrolls to or selects the next image in the group, the system can perform blocks S240 and S242 for all remaining images in the group. Alternatively, the system can implement the aforementioned methods and techniques to propagate the last view window of the first image to the images in the group in real time as the user pans forward and backward through the other images within the user interface.

[0375] 2.9 Composite Image

[0376] A variation of the second method S200 includes block S252, which includes virtually aligning the images in the set by their coordinate systems, reducing the opacity of the images, and overlaying the images to form a composite image. Generally speaking, in this variation, the system can: virtually stack two (or more) images from the set of images with their similar features or coordinate systems aligned based on similar features; reduce the opacity of the images to form a composite image; and display the composite image within a user interface. Thus, when viewing the composite image, a user can view deviations in position and geometry of similar components (e.g., housings, subassemblies, parts, subparts) of the assembly unit represented in the images relative to the common reference feature.

[0377] For example, in block S252, the system may: set a first opacity for a first subregion of a first image; set a second opacity for a second subregion of a second image; overlay the second subregion on the first subregion to generate a composite image; and display the composite image within a user interface. When generating the composite image, the system may apply a static opacity, such as 50% opacity, to each image. Alternatively, the system may enable a user to dynamically adjust the opacity of the images represented in the composite image and then update the composite image reproduced in the display accordingly. For example, the system may: present a slider bar adjacent to the composite image displayed in the user interface; adjust the first opacity of the first image based on a change in the position of a slider on the slider bar; adjust the second opacity of the second image as an inverse function of the first opacity; and refresh the composite image accordingly.

[0378] The system may implement similar methods and techniques to align and combine two or more complete images from the set of images into a composite image.

[0379] In another implementation, the system generates a composite image from an image of a real assembly unit and an image of a graphical model representing the assembly unit. In this implementation, by aligning the image of the real assembly unit with the image of the graphical model representing the assembly unit within a single composite image and then reproducing the composite image within the user interface, the system can enable a user to quickly visually distinguish differences in component positions and orientations between the real assembly unit and the nominal representation of the assembly unit defined in the graphical model. For example, the system can: obtain a virtual three-dimensional computer-aided drafting (“CAD”) model representing a first assembly unit; generate a two-dimensional CAD image of the CAD model in an orientation and perspective approximately similar to the orientation and position of the first assembly unit represented in the first image; locate a third virtual origin at a third feature (similar to the first feature on the first assembly unit) in the CAD image, for example by implementing methods and techniques similar to those of block S240 above; project the geometry and position of a first sub-region of the first image onto the virtual CAD model according to the third virtual origin to define a third image, for example by implementing methods and techniques similar to those of block S242 above; and then display a translucent form of the third image over the first sub-region of the first image within the user interface. Thus, in this example, the system can align the CAD image with the first image in rotation and translation by real features on the real assembly unit represented in the first image and graphical features representing real features in the CAD model.

[0380] Optionally, the system can implement similar methods and techniques to: generate a CAD image; project a view window from the first image onto the CAD image to define a sub-region of the CAD image similar to the first sub-region of the first image; and display the sub-region of the CAD image within the user interface independently of the first image, for example when the user scrolls from the first image to the CAD image while a new view window specification is in effect.

[0381] 2.10 An Assembly Unit at Different Assembly Stages

[0382] In a variation, the system implements similar methods and techniques to maintain a view window across a set of images of a single assembly unit over an entire series of assembly stages. For example, in block S120, the system can assign a virtual origin to the first image based on features corresponding to the largest physical body shown in the first image (such as the corner of a PCB, the corner of a rectangular housing, or a vertical side). In this example, the system can identify the same features in other images of the assembly unit at different assembly stages and assign similar virtual origins to these other images.

[0383] In this variation, the second method S200 may include: in block S210, displaying, within a user interface, a first image of an assembly unit in a first assembly stage, the first image being recorded at a first optical inspection station; in block S220, positioning a first virtual origin in the first image at a feature on the assembly unit represented in the first image; in response to receiving a zoom input at the user interface, in block S212, displaying, within the user interface, a first sub-region of the first image; in block S230, storing the geometry and position of the first sub-region of the first image relative to the first virtual origin; in block S240, identifying a feature on the assembly unit in a second image of the assembly unit in a second assembly stage; in block S240, positioning a second virtual origin in the second image based on the feature; in block S242, defining a second sub-region of the second image based on the geometry and position of the sub-region of the first image and the second virtual origin; and in response to receiving a command at the user interface to advance from the first image to the second image, in block S250, displaying, within the user interface, the second sub-region of the second image.

[0384] For example, the system may: obtain a first digital photographic image of a first assembly unit, the first digital photographic image being recorded by a first optical inspection station at a first position along an assembly line; normalize the first digital photographic image as described above to form a first image; obtain a second digital photographic image of the first assembly unit, the second digital photographic image being recorded by a second optical inspection station at a second position along the assembly line; and normalize the second digital photographic image to form a second image. Then, the system may implement the above methods and techniques to define similar sub-regions of the first and second images (and other images of the first assembly unit), and sequentially display these sub-regions as the user pans through these images.

[0385] In particular, in this variation, the system may implement the above methods and techniques to display an expanded view of the same physical location of a single assembly unit from a sequence of images recorded at various assembly stages of the assembly unit. By aligning images of an assembly unit at different assembly stages by a common feature and sequentially displaying these images in a user interface in response to a scroll or pan input entered by the user, the system may enable the user to view changes to the assembly unit over time (e.g., along an assembly line), the view windows of these individual images being locked to a common reference feature included in these images.

[0386] 2.10.1 Masked Alternative Reference Features

[0387] In this variation, if, for example, due to a component mounted on the assembly unit obscuring between the capture of the first image and the capture of the second image, the reference feature selected in the first image to define the virtual origin in the first image is not visually available in the second image of the same unit, the system can: select an alternative feature that is visually available in both the first image and the second image on the assembly unit when the user scrolls from the first unit to the second unit; and then re-define the virtual origin (or assign a secondary or alternate virtual origin) for the first and second images of the assembly unit.

[0388] In particular, the system can: align the first image (or a sub-region of the first image) of the assembly unit with the second image (or a sub-region of the second image) by a common feature represented in both the first image and the second image; locate a third virtual origin in the second image at a second feature on the assembly unit represented in the second image; store the geometry and position of a second sub-region of the second image relative to the third virtual origin; identify the second feature on the assembly unit in a third image; locate a fourth virtual origin in the third image based on the second feature; define a third sub-region of the third image based on the geometry and position of the second sub-region of the second image and the fourth virtual origin; and display the first, second, and third sub-regions of the first, second, and third images as the user progresses through the assembly stage of the assembly unit at the user interface. For example, the system can align the first sub-region of the first image with the second sub-region of the second image by a corner of a PCB within the assembly unit shown in both the first and second sub-regions of the first and second images. In this example, before the third image is recorded, the housing is mounted on the assembly unit, thus obscuring the PCB. Therefore, to align the third image with the second image, the system can detect an edge of the housing of the assembly unit shown in both the second image and the third image and align the second image and the third image based on the edge of the housing.

[0389] 2.10.2 Transparent Composite View

[0390] In this variation, the system can implement the above methods and techniques to assemble two or more images (or sub-regions of two or more images) of an assembly unit at different assembly stages into a single composite image. For example, the system can: set a first opacity for a first sub-region of a first image of the assembly unit at a first assembly stage; set a second opacity for a second sub-region of a second image of the assembly unit at a second (e.g., later) assembly stage; align the first and second sub-regions of the first and second images by common features of the assembly unit represented in both the first and second sub-regions; merge the first and second sub-regions into a composite image; and then display the composite image within a user interface. In this example, the system can assemble multiple (e.g., all) images of an assembly unit into a static composite image and reproduce the static composite image within the user interface. Optionally, the system can assemble these images into a dynamic composite image. For example, the system can initially display the composite image with the first image of the assembly unit in the first assembly state being shown at 100% opacity and all other images at 0% opacity; as the user scrolls through the entire composite image, the system can decrease the opacity of the first image and increase the opacity of the second image of the assembly unit in the second assembly state; once the second image is shown at 100% opacity and as the user continues to scroll through the entire composite image, the system can decrease the opacity of the second image and increase the opacity of the third image of the assembly unit in the third assembly state; and so on, until the last image of a set of images of the assembly unit is shown at full opacity.

[0391] 3. Optical Measurement

[0392] As Figure 6 shown, a third method S300 for automatically generating common measurements across multiple assembly units includes: detecting a first set of features in a first assembly unit shown in a first image captured by an optical inspection station in block S310; displaying the first image and a set of curves regarding the set of features in the first image at a user interface in block S320; and generating a measurement specification for a specific feature in the assembly unit imaged by the optical inspection station based on a curve manually selected from the set of curves within the user interface in block S330. The third method S300 also includes for each image in a set of images captured by the optical inspection station (the set of images including the first image): identifying a specific feature in the assembly unit shown in the image based on the measurement specification in block S340; mapping a distorted measurement space onto the image in block S344; and calculating the true size of the specific feature based on the geometry of the feature within the distorted measurement space in block S344. Finally, as Figure 7 and 8As shown, the third method S300 may include generating, in block S350, a graphical curve of the true dimensions of a specific feature in the assembly unit imaged by the optical inspection station.

[0393] A variation of the third method S300 for automatically generating a common measurement across multiple assembly units includes: displaying, in block S320, a first image within a user interface, the form of the first image being recorded at the optical inspection station; receiving, in block S310, a manual selection of a specific feature in the first assembly unit represented in the first image; receiving, in block S330, a selection of a measurement type for the specific feature; and extracting, in block S344, a first true dimension of the specific feature in the first assembly unit from the first image according to the measurement type. The third method S300 further includes, for each image in a set of images: identifying, in block S340, a feature in the assembly unit represented in the image that is similar to the specific feature in the first assembly unit; and extracting, in block S344, the true dimension of the feature in the assembly unit from the image according to the measurement type. The third method S300 further includes aggregating, in block S350, the first true dimension and a set of true dimensions extracted from the set of images into a digital container.

[0394] Another variation of the third method S300 includes obtaining a set of images in block S310, and for a first image in the set of images: displaying the first image within the user interface, the form of the first image being recorded at the optical inspection station, and displaying a first true size together with the first image within the user interface in block S320; receiving a manual selection of a specific feature in a first assembly unit represented in the first image in block S310; determining a measurement type of the specific feature in block S330; and extracting a first true size related to the specific feature in the first assembly unit from the first image according to the measurement type. The third method S300 further includes, for a second image in the set of images: automatically identifying a second feature in a second assembly unit represented in the second image in block S340, the second feature in the second assembly unit being similar to the specific feature in the first assembly unit; and extracting a second true size related to the second feature in the second assembly unit from the second image according to the measurement type in block S344. In this variation, the third method S300 further includes, for a third image in the set of images: automatically identifying a third feature in a third assembly unit represented in the third image in block S340, the third feature in the third assembly unit being similar to the specific feature in the first assembly unit; and extracting a third true size related to the third feature in the third assembly unit from the third image according to the measurement type in block S340. The third method S300 further includes: in response to a selection of the second image at the user interface, displaying the second image and the second true size within the user interface in block S320; and in response to a selection of the third image at the user interface, displaying the third image and the third true size within the user interface in block S320.

[0395] 3.1 Application

[0396] Generally, the system can execute the blocks of the third method S300: to receive a selection of a true feature in a first unit from a first image of the first unit; define a call to calculate the true size (e.g., length, radius, parallelism, etc.) of the feature from the first image; propagate the call across other images of similar units; and automatically calculate the true sizes of similar features across multiple similar units from the corresponding images of the assembly units. Then, the system can assemble these true sizes of one feature type from multiple units into a graphical plot, histogram, table, trend line, or other graphical or numerical representation of the true sizes of the features across the set of units. The system also calculates the true size of a feature in a unit by implementing machine vision techniques (e.g., edge detection) to identify features in an image of the assembly unit, mapping a two-dimensional or three-dimensional measurement space to the image to compensate for optical distortion in the image, and then calculating the true size of the feature based on the position of the feature in the image relative to the measurement space.

[0397] Therefore, the system can execute the blocks of the third method S300 to retrospectively and automatically configure the measurement of a batch of units from the images of the assembly units after capturing the images, thereby eliminating the need to configure an optical inspection station on the assembly line before assembling the units on the assembly line. The system can also execute the blocks of the third method S300 to enable one or more users to create and access new measurements of real unit features from old images of the units, without the need to capture new images of the assembly units and without the need to manually select the same features for the measurement of images of multiple similar units. The system can similarly execute the blocks of the third method S300, for example, locally at the optical inspection station, in a remote server, or on the user's computing device (such as a smartphone) in real time.

[0398] The blocks of the third method S300 can be executed locally by a computing device such as a smartphone, a tablet, or a desktop computer. The system can include a display and reproduce a user interface on the display to receive the selection of features in the image for measurement and present the true size results to the user. The blocks of the third method S300 can additionally or optionally be executed by a remote computer system such as a remote server, which is connected to the local computing device (such as a smartphone, a tablet, or a desktop computer) through an interface to receive the selection of features in the image for measurement and present the true size results to the user. However, any other local or remote computing device, computer system, or computer network (hereinafter referred to as "system") can execute the blocks of the third method S300.

[0399] The third method S300 is herein described as being implemented by the system to disseminate measurement specifications of dimensional values (e.g., length, radius, parallelism, etc. in metric or imperial units). However, the third method S300 can additionally or optionally be implemented to define part presence specifications, disseminate part presence specifications across images of multiple units, and confirm whether the components specified in the part presence specifications are present in each image. Similarly, the third method S300 can be implemented to define categories of text specifications, color specifications, markings (such as production batch codes) specifications, or any other type of specifications, disseminate these specifications across images of multiple units, and confirm whether the text strings, colors, markings, or other features specified in the part presence specifications are present in each image.

[0400] 3.2 Image

[0401] Block S320 of the third method S300 recites displaying a first image within the user interface. Generally, in block S320, the system can implement the methods and techniques described above in block S210 to obtain the first image from the database and present the first image to the user through the user interface.

[0402] As described above in block S210, the system can also: normalize (or "flatten", "undistort") the first image and other images stored in the image database (or "corpus"); aggregate a set of related images, such as images of various assembly units of the same type and in the same assembled state or a set of images representing a first (i.e., single) assembly unit in various stages of assembly.

[0403] 3.3 Feature Selection

[0404] Block S310 of the third method S300 recites receiving a manual selection of a specific feature in the first assembly unit represented in the first image. Generally, in block S310, the system is interfaced with the user via a user interface to receive a selection of a specific feature or a specific group of features, and the system then extracts dimensions from the specific feature or specific group of features in block S344.

[0405] 3.3.1 Feature Space and Vector-Based Selection

[0406] In one implementation, the system: implements computer vision techniques to identify features of the first assembly unit represented in the first image; generates a feature space containing vectorized points, lines, curves, regions, and / or planes representing these features; and overlays the first image with the feature space, as described above. In particular, when the user selects an image of a unit captured by an optical inspection station for insertion for measurement, in block S310, the system can implement machine vision techniques to automatically detect features of the assembly unit shown in the first image. For example, the system can implement edge detection techniques to identify corners (e.g., points), edges (e.g., lines, curves), and surfaces (e.g., regions, planes) in the first image. In block S320, to guide the user in selecting one or more features in the first image for measurement, the system can: generate a feature space specific to the first image that contains vectorized points, curves, regions, and / or planes aligned with the points, lines, and surfaces detected in the first image; and then reproduce the first image and the feature space placed on the first image in the user interface, as Figure 6 shown. The system can then receive from the user via the user interface a manual selection of a specific vector (or group of vectors) from a first set of vectors included in the first feature space, and then identify the specific feature (or group of features) corresponding to the specific vector selected by the user.

[0407] 3.3.2 Pixel-Based Feature Selection

[0408] Optionally, the system may be interfaced with a user interface to receive a selection of pixels from the first image and implement the methods and techniques described above to select the particular feature in the first image that is closest to or otherwise corresponds to the pixel from a set of features. For example: when viewing the first image within the user interface, the user may navigate the cursor to a pixel near a desired corner feature, near a desired edge feature, or on a desired surface and select that pixel; as described above, the system may then compare the pixel selection to the feature space specific to the first image to identify the particular feature that is closest to the selected pixel.

[0409] As described above, the system may also prompt the user to select multiple pixels closest to a desired corner along a desired edge or on a desired surface represented in the first image; the system may then compare these selected pixels to the feature space to select a corner, line (or curve), or region that best fits the set of selected pixels. However, in block S310, the system may be interfaced with the user in any other way to receive a selection of a particular feature from the first image. As described below, the system may implement similar methods and techniques to receive selections of multiple different features from the first image.

[0410] The system may implement subsequent blocks of the third method S300 to define measurement specifications for the set of images based on the feature, extract the true dimensions of the feature from the image, and populate the measurement specifications across the other images in the set.

[0411] 3.4 Measurement Specification

[0412] Block S330 of the third method S300 recites receiving a selection of a measurement type for a particular feature. Generally, in block S330, the system generates a measurement type that defines the feature selected in block S310 and characterizes the measurement specifications of the particular feature selected from the first image.

[0413] As described above, the system may receive from the user a selection of one or more vectorized curves in the feature space. For example, from the vectorized curves contained in the feature space overlaid on the first image, the user may select a vectorized point, an intersection of two vectorized curves, a single vectorized curve, two non-intersecting vectorized curves, or a region enclosed by one or more vectorized curves. The system may populate a measurement type menu within the user interface with various measurement types such as distance (e.g., corner-to-corner), length (e.g., end-to-end or edge length), radius (or diameter), flatness, parallelism, roundness, straightness, line profile, surface profile, perpendicularity, angle, symmetry, concentricity, and / or any other measurement type of a particular feature in the first image; the user may then select a measurement type from the menu for the selected point, intersection, curve, and / or region in the feature space.

[0414] Based on the type of feature selected by the user, the system can also filter, sort, and / or suggest measurement types from a set of supported measurement types. For example, when a single line (e.g., a substantially straight curve) is selected in block S310, the system can predict length-type measurements and can accordingly enable the length-type measurement types in the measurement type menu. When an arc is selected in block S310, the system can enable total arc length measurement, radius measurement, and diameter measurement in the measurement type menu. When a region is selected in block S310, the system can enable total area measurement and perimeter measurement in the measurement type menu. When a point and a curve are selected in block S310, the system can enable closest distance measurement and orthogonal distance measurement in the measurement type menu. When a first curve and a second curve are selected in block S310, the system can enable closest distance measurement, angle measurement, and gap profile measurement (e.g., gap distance as a function of the lengths along the first curve and the second curve) in the measurement type menu. When three points are selected in block S310, the system can enable measurement in the measurement type menu for calculating the minimum circle formed by the three points. However, the system can support any other predefined or user-defined (e.g., customized) measurement types. The system can also receive a selection of measurement types or can otherwise automatically predict the measurement types for specific features.

[0415] From a specific feature selected from the first image (e.g., an original feature in the first image or a vectorized point, curve, and / or region, etc. in a feature space specific to the first image), in block S330, the system can generate measurement specifications for the set of images. For example, in block S330, the system can define a feature window that encloses the specific feature in the first image and store the position and geometry of this feature window (e.g., relative to the origin of the first image or relative to the upper left corner of the first image) in the measurement specifications; as Figure 6 shown, in block S340, the system can project the feature window onto the other images in the set to identify features in these other images in the set that are similar to the specific feature selected from the first image. In particular, when processing the set of images according to the measurement specifications in block S340, the system can project the feature window onto each image in the set and then can scan the regions of these images delimited by the feature window to find features that are similar to the feature selected in block S310. In this implementation, the system can implement the methods and techniques described above in the second method S200 to align the feature window defined at the first image with the other images in the set.

[0416] The system can also characterize a specific feature and store the characterization in the measurement specification. For example, the system can: implement template matching or pattern recognition techniques to characterize the specific feature as one of an arc, spline, circle, or line; write this characterization of the specific feature to the measurement specification in block S330; and apply the characterization to other images in the group in block S340 to identify the same type of feature in these other images. The system can also: calculate the true size or pixel-based size of the specific feature; store this size in the measurement specification; and detect in block S344 similar features in the remaining images in the group that exhibit a similar true size or pixel-based size, for example, within a tolerance of ±2%. Similarly, the system can: prompt the user to enter the nominal size and the dimensional tolerance of the nominal size of the specific feature, as Figure 7 and 8 shown; or extract the nominal size and the dimensional tolerance from the CAD model of the first assembly unit as described below; and identify features similar to the specific feature in other images in the group based on the nominal size of the feature.

[0417] The system can also prompt the user to enter the name of the measurement (e.g., "antenna_height_1"), the description of the measurement (e.g., "antenna height"), the label or search term for the measurement (e.g., "John_Smiths_measurement_set", "RF group", "DVT", or "EVT", etc.), and / or other textual or numerical data for the measurement. Then, the system can store this data in the measurement specification, as Figure 6 and 9C shown. For example, the system can enable the user to switch such labels within the user interface to access and filter points represented in a graph or chart of the true sizes of similar features read from the images in the group according to the measurement specification. Similarly, the system can enable another user to search the measurement specification by entering one or more of these labels or other terms via a search window in another instance of the user interface in order to access the measurement specification for the group of images or access the application of the measurement specification across another group of images. Thus, the system can support multiple users: to apply general, group-specific, and / or user-specific measurement specifications across various groups of images; to access data extracted from a group of images according to general, group-specific, and / or user-specific measurement specifications; and to access measurement specifications configured by other users.

[0418] However, in block S330, the system can collect any other relevant information from the user or extract any other relevant data from the first image to generate the measurement specifications for the set of images. The system can then apply the measurement specifications across the images in the set in blocks S340 and S344 to identify similar features in the assembly units represented in these images and directly extract the true dimensions of these similar features from these images.

[0419] 3.5 Similar Feature Detection

[0420] Block S340 of the third method S300 describes identifying, for each image in a set of images, features in the assembly unit represented in the image that are similar to a specific feature in the first assembly unit. (Block S340 can similarly describe identifying a second feature in a second assembly unit represented in a second image, the second feature in the second assembly unit being similar to the specific feature in the first assembly unit.) Generally, in block S340, the system: scans the images in the set to find features that are similar to a specific feature selected from the first image, e.g., by implementing the methods and techniques described above in the second method S200, such as features located in similar positions and exhibiting a similar geometry (e.g., true or pixel-based dimensions, feature type) to the specific feature selected from the first image; and repeats the process for the remaining images in the set to identify a collection of similar features in the assembly units represented across the set of images. Then, in block S344, the system can directly extract the true dimensions of these features from these images and assemble these dimensions into a curve, chart, or statistic representing the dimensional variation of similar features (e.g., length, width, part radius; relative position of parts; gap between parts; etc.).

[0421] To calculate the true dimensions of similar features across a set of images of an assembly unit in the same or similar assembly stages in one or more configurations according to a single measurement specification configured by the user, the system can scan each image in the set to find features that are similar (e.g., substantially similar, equivalent, corresponding) to the feature selected by the user and specified in the measurement specification. For example, for each image selected according to the measurement specification for true dimension calculation, the system can implement the above methods and techniques to detect features in the image and then, in block S340, select in the image a feature that best matches the relative dimensions, geometry, position (e.g., relative to other features represented in the image), color, surface finish, etc. of the specific feature selected from the first image and specified in the measurement specification. The system can then calculate the dimensions of the similar features for each image in the set in block S344, as described below.

[0422] 3.5.1 Window Scanning

[0423] In one implementation, the system: defines a feature window that encloses a specific feature, is offset from the specific feature, and is positioned according to the origin of the first image; and stores the feature window in the measurement specification in block S330 as described above. For example, the system can define the feature window relative to the global origin of the image (e.g., the upper left corner of the first image). Optionally, the system can implement computer vision techniques to detect the perimeter of the first assembly unit in the first image, define the origin on the first assembly unit in the first image (e.g., at the upper left corner of the first assembly unit), and define the position of the feature window relative to the origin based on the assembly unit. The system can also implement a preset offset distance (e.g., 50 pixels), and define the geometry of a future window that encloses the specific feature and is offset from the specific feature by the preset offset distance. For example, for a specific feature that defines a corner, the system can define a circular feature window with a diameter of 100 pixels; for a specific feature that defines a linear edge, the system can define a 100-pixel-wide rounded rectangular feature window with corners having a radius of 50 pixels shown and offset 50 pixels from the proximal end of the linear edge.

[0424] Then, the system can position the feature window within the image according to the global origin of the image (e.g., the upper left corner of the image). Optionally, the system can repeat the above process to define the origin based on the assembly unit within the image and position the feature window in the image according to the origin based on the assembly unit. Then, the system can: scan the image region delimited by the feature window to identify a finite set of features in the image; and compare the geometries and dimensions of the features in the finite set of features with the representation of the specific feature stored in the measurement specification to identify a feature in the image that best approximates (e.g., "is similar to") the specific feature from the first image. The system can repeat this process for each remaining image in the set of images.

[0425] 3.5.2 Feature Matching in Feature Space

[0426] In another implementation, the system generates a first feature space for the first image in block S310, labels a specific vector representing the specific feature in the first feature space, and stores the first feature space in the measurement specification in block S330. Then, the system can implement similar methods and techniques to identify a set of features in the images in the set of images and generate a feature space that contains a set of vectors representing the set of features in the images. Then, the system can: align the feature space of the image with the first feature space of the first image; identify the vector in the set of vectors that is closest in position and geometry to the specific vector in the first set of vectors; and label the feature in the image corresponding to the vector as similar to the specific feature in the first image. The system can repeat this process for each remaining image in the set of images.

[0427] However, the system can implement any other method or technique to identify features of the assembly units represented in the set of images that are similar to the specific features of the first assembly unit selected from the first image. Additionally, in block S310, the system can repeat the foregoing process for each of the plurality of distance features selected from the first image.

[0428] 3.5.3 Feature Confirmation

[0429] In a variation, before repeating the process to identify similar features in other images in the set, the system implements the above methods and techniques to receive the following confirmation from the user: the identification of the second feature in the second image in the set of images is similar to the specific feature selected from the first image. For example, once the measurement specification is defined in block S330, the system can: perform a feature selection routine to identify a second feature in the second image that is predicted to be comparable (i.e., similar) to the specific feature selected from the first image; and store the steps of the feature selection routine or a characterization of the feature selection routine in a memory (e.g., in the measurement specification). Before repeating the feature selection routine for other images in the set, the system can: display the second image in a user interface; indicate the second feature within the second image; and prompt the user via the user interface to confirm that the second feature is similar to the specific feature. If the user indicates that the second feature is incorrect, the system can repeat the feature selection routine to select an alternative feature from the second image and repeat the process until the user indicates that the correct feature has been selected. The system can additionally or alternatively prompt the user to manually indicate the correct feature in the second image, and the system can update the feature selection routine accordingly. However, in response to receiving a manual confirmation from the user via the user interface of a second feature that is similar to the specific feature, the system can: perform the feature selection routine at a third image in the set to identify a third feature in the third image that is similar to the specific feature; and perform the feature selection routine at other images in the set to identify similar features in those other images.

[0430] 3.6 Measurement Propagation

[0431] Block S344 of the third method S300 recites: extracting a first true dimension of a specific feature in the first assembly unit from the first image according to a measurement type; and for each image in a set of images, extracting a true dimension of a feature in the assembly unit from the image according to the measurement type. Generally, once similar features are identified in each image in the set of images in blocks S310 and S340, the system directly extracts the dimension of each of these features from their corresponding images.

[0432] In addition, once the dimensions of features are extracted from an image of the assembly unit, the system can reproduce an indication of the features and their dimensions within the user interface, for example, on or adjacent to the image. In particular, in response to selecting a first feature from a first image at the user interface, the system can display, within the user interface and together with the first image (e.g., on or adjacent to the first image), the first true dimension of the first feature; in response to selecting a second image of a second assembly unit, the system can display, within the user interface, the second image and the second true dimension of a second feature similar to the first feature; and so on.

[0433] 3.6.1 True Dimensions from Original Image

[0434] In a variation, the system projects a dimensional space onto a first ("flattened") image, extracts a first true dimension of a specific feature from the first image based on the position and measurement type of the specific feature relative to the dimensional space, and repeats the process for other images in the set.

[0435] In this implementation, the system can flatten an original digital photographic image of a first assembly unit and present the flattened first image to the user via the user interface for selection of a specific feature. Once the specific feature is selected, the system can project the specific feature from the flattened first image onto the first digital photographic image to identify the specific feature in the original digital photographic image. The system can then map a distorted measurement space onto the first digital photographic image to prepare for extraction of the true dimension of the specific feature from the digital photographic image. Generally, in this variation, to accurately (i.e., precisely and reproducibly) calculate the true dimension of a feature of the assembly unit represented in the flattened image, the system can project the distorted measurement space onto the corresponding digital photographic image and extract the dimension of the feature from the digital photographic image based on the position of the feature relative to the distorted measurement space. In particular, rather than extracting the true dimension from the flattened image, which may result in loss of data on the original digital photographic image, the system can map the distorted measurement space onto the corresponding digital photographic image to compensate for optical distortion (such as perspective distortion) in the digital photographic image while also preserving the data contained in the image.

[0436] In one implementation, the system generates a virtual measurement space that represents a plane in the real space at a specific distance from the camera in the optical inspection station. The optical inspection station re - encodes digital photographic images but "distorts" (i.e., "distorts") them in two or three dimensions to represent the optical distortion in the digital photographic images generated by the optics in the camera. In one example, after capturing a digital photographic image of an assembly unit, the optical inspection station can label the digital photographic image at the moment the digital photographic image is recorded with the zoom level, focus position, aperture, ISO, and / or other imaging parameters implemented by the optical inspection station. In this example, to calculate the dimensions of features in the digital photographic image, the system can: extract these imaging parameters from the metadata stored in the digital photographic image; calculate a reference plane on which the real features of the assembly unit appear in the real space relative to a real reference (e.g., a datum on the optical inspection station); and then generate a virtual measurement space based on the imaging parameters stored with the digital photographic image. The virtual measurement space contains a set of X and Y grid curves offset by a virtual distance corresponding to a known real distance on the real reference plane, as Figure 6 shown. Then, the system can calculate the length, width, radius, etc. of the features shown in the digital photographic image by interpolating between the X and Y grid curves in the virtual measurement space overlaid on the digital photographic image.

[0437] In the foregoing example, the system can select pixels or groups of pixels at each end of a feature in the digital photographic image that resembles a specific feature, project these pixels onto the distorted measurement layer, interpolate the real position of each projected pixel or group of pixels based on its position relative to the X and Y grid curves in the measurement space, and then calculate the real length of the feature in the real unit (or the distance between two features) based on the differences between the interpolated real positions of the pixels or groups of pixels. In this example, the system can select pixels or groups of pixels at each end of a feature in the digital photographic image corresponding to a feature defined in the measurement specification, generate a virtual curve representing a real straight line in the measurement space and passing through the pixels or groups of pixels, and then calculate the straightness of the feature in the assembly unit from the variation between the pixels corresponding to the feature in the digital photographic image and the virtual curve in the measurement space. Thus, the system can generate a distorted measurement layer from a standard calibration grid based on a calibration datum in the digital photographic image or based on any other general optical inspection - station - specific imaging or digital - photographic - image - specific parameters.

[0438] However, the system can generate any other form of measurement layer (or multi-dimensional measurement space) in block S344 and can apply the measurement layer to the digital photographic image in any other way in block S344 to calculate the true dimensions of features on the assembly unit according to the parameters defined in the measurement specification. When the corresponding digital photographic image is reproduced with the user interface, the system can also reproduce a virtual form of the measurement layer on the corresponding digital photographic image, for example, in the form of a distorted grid overlay.

[0439] 3.6.2 True Dimensions from Undistorted Image

[0440] In another implementation, the system directly extracts the true dimensions from the flattened image (described above). For example, as described above, when calculating the homography transformation for flattening the digital photographic image recorded at the optical inspection station, for example, based on the reference image recorded at the optical inspection station, the system can calculate the scalar coefficient that associates the length of digital pixels in the flattened image with the true dimensions (i.e., the length values in the real space). In block S344, to extract the true dimensions of features from the image, the system can: count the number of pixels spanning the feature; and multiply that number of pixels by the scalar coefficient to calculate the true dimensions of the feature. However, the system can implement any other method or technique to extract the true dimensions of the true features on the assembly unit from the flattened image of the assembly unit. The system can implement these methods and techniques for each image in the set of images to calculate the true dimensions of similar features spanning the set.

[0441] (The system can additionally or alternatively implement the methods and techniques described herein to calculate the dimensions of the assembly unit in only one image of the assembly unit based on the measurement specification (e.g., rather than propagating the measurement specification across all images or a subset of images). For example, the system can implement these methods and techniques to calculate a one-time measurement based on pixel-to-pixel selection input by the user onto a single image.

[0442] 3.7 Access and Analysis

[0443] Block S350 of the third method S300 recites aggregating the first true dimensions and a set of true dimensions extracted from the set of images into a digital container (e.g., a virtual visual representation or a digital file). Generally, in block S350, the system aggregates the true dimensions representing similar (e.g., analogous) features of a set of assembly units from the set of images into a visual or statistical (e.g., digital) representation of the variation of the feature across the set of assembly units.

[0444] 3.7.1 Curves and Charts

[0445] In one implementation, the system can compile real measurements of similar features in the assembly units across the set of image representations into a graphical plot. For example, the system can aggregate the set of real dimensions into a virtual histogram that includes a set of discrete percentile ranges (e.g., 0-10%, 10-20%, 20-30%, etc.) across the set of real dimensions, and then render the virtual histogram within the user interface, such as Figure 9A In this example, in response to selecting a particular percentile range in the set of discrete percentile ranges (or placing a cursor over a particular percentile range), the system can obtain an exemplary image in the set of images that represents the particular percentile range and reproduce the exemplary image above or next to the virtual histogram within the user interface.

[0446] In another implementation, the system may convert the set of real dimensions into a graph of real dimensions versus time (e.g., as recorded along an assembly line for time and digital photographic images of assembled units) or versus serial number, such as Figure 9B As shown; and then displaying the graph in a user interface. The system can also calculate a trend line or best fit line for the graph and, based on this trend line, predict deviations from the nominal size of the feature that are outside of dimensional tolerance. For example, the system can perform the aforementioned methods and techniques in substantially real time as assembly units are assembled along an assembly line and as digital photographic images of these units are received from optical inspection stations arranged along the assembly line. In this example, the system can: generate a graph of the true size of a particular feature versus the serial number across a series of assembly units based on data extracted from these images; and (re)calculate the trend line of the graph after each additional digital photographic image is received from the optical inspection station. If the trend line exhibits a positive slope (per unit time or per assembly unit) exceeding a threshold slope (e.g., once a threshold number of images of the assembly unit have been processed), the system can automatically generate a flag for the assembly line and prompt a user (or other engineer or entity associated with the assembly line) to review the processes associated with the corresponding feature at the assembly line to preempt deviations from a specified size for this particular feature that are outside of dimensional tolerance. Thus, the system can extract the dimensions of specific features from an image, infer trends in the dimensions of the specific features, and selectively prompt a user or other entity to study the assembly line in (near) real time in order to achieve improved yield from the assembly line.

[0447] In the foregoing implementation, the system can be connected to the user through an interface via a user interface to configure the measurement specifications of the assembly unit type or the assembly line before any assembly unit is assembled or imaged at the assembly line. For example, the system can: obtain a three-dimensional CAD model of the assembly unit, as described above in the second method S200; display a two-dimensional or three-dimensional CAD image of the CAD model within the user interface in block S320; receive a user selection of a specific feature directly from the CAD image in block S310; and configure the measurement specifications of the specific feature based on the selection and other data extracted from the CAD model (e.g., the nominal dimensions and dimensional tolerances of the specific feature, feature type, component type, and configuration, etc.) in block S330. Then, the system can extract the true dimensions of the features corresponding to the specific features selected from the CAD model from each image of the components received from the optical inspection station in (near) real time according to the CAD-based measurement specifications.

[0448] Optionally, the system can retrospectively apply the measurement specifications to a corpus of images previously recorded and stored in a database, e.g., to enable the user to access the dimensional statistics of specific features in a batch of assembly units from a previous product model when designing the next product model. For example, the system can extract the average dimension, standard deviation, deviation from the nominal dimension, instances of deviation from the nominal dimension exceeding a preset dimensional tolerance, or any other statistics related to the true dimensions of similar features across a set of (similar) assembly units represented in a set of images. Then, the system can present this graphical and / or numerical data to the user through the user interface. However, the system can generate any other type of curve graph, graph, or chart of the dimensions of a specific feature across the set of assembly units, or extract any other statistics from these dimensional data.

[0449] Optionally, the system can encapsulate these dimensions into a spreadsheet or other digital file or database; then, the user can download the file for manipulating these dimensional data in another program. However, in block S350, the system can encapsulate these dimensional data into any other digital container.

[0450] 3.7.2 Outliers

[0451] In another implementation, the system detects dimensional outliers in real-time or asynchronously based on images stored in a database when, for example, digital photographic images are received from an optical inspection station and marks individual assembly units accordingly. For example, in the implementation described above where the system generates a histogram of the dimensions of similar features across the set of images, the system can: identify a subset of images representing features exhibiting dimensions that fall within an upper limit (e.g., top 10%) of the dimensions represented in the histogram; and mark the serial numbers stored in the image metadata of these assembly units or extracted directly from the visual data contained in these images, as Figure 9A shown. Similarly, the system can: compare the dimensions of similar features extracted from the set of images directly with the nominal dimensions and dimensional tolerances associated with the feature, for example, by manual input or by extracting this data from a CAD model or engineering drawing, as described above; and mark the serial numbers of the assembly units containing features that exhibit the feature differing from the nominal dimension by more than the dimensional tolerance, as Figure 9B shown.

[0452] Thus, in this implementation, the system can: access the target dimension of a specific feature and the dimensional tolerance of the target dimension of the specific feature, for example, by prompting the user to manually enter these values when configuring measurement specifications or by extracting these values from a CAD model or engineering drawing. Then, the system can: mark the serial number of a second assembly unit that includes a second feature similar to the specific feature characterized by an actual dimension that differs from the target dimension by more than the dimensional tolerance.

[0453] When receiving images from an optical inspection station deployed along the assembly line, the system can perform this process (near) in real-time. In the foregoing example, the system can: configure measurement specifications at the selection of the specific feature in the first assembly unit based on a first image; identify a second feature similar to the first feature in a second assembly unit represented in a second image recorded at the assembly line at a second time subsequent to the first time; extract a second actual dimension of the second feature in the second assembly unit from the second image according to the measurement specifications; and transmit, at approximately the second time, an electronic notification containing the fourth assembly unit via a user interface or another computing device associated with the user.

[0454] In this implementation, the system can also access the dimensional range of features similar to the specific feature selected in block S310 that are associated with a component's failure, where the feature is, for example, stored in a database or manually entered by a user during the current image viewing period. Upon receiving the second image of the second assembly unit, the system can: identify the second assembly unit that contains a second feature similar to the specific feature and is characterized by a second true dimension that falls within the dimensional range associated with the component failure; and then provide a prompt to inspect the second assembly unit to an electronic account associated with the user, for example, (near) real-time as the second assembly unit passes through the assembly line. The system can implement similar methods and techniques to asynchronously flag assembly units that contain features with dimensions within the failure range. For example, if testing of a subset of the assembly units assembled after the production of a larger batch of assembly units later indicates an association between the dimension of a specific feature and the failure, the system can flag other units in the larger batch that were not tested for the assembly units, which may fail due to exhibiting similar features with dimensions within the failure range.

[0455] 3.7.3 Multiple True Dimensions

[0456] In Figure 9C In one implementation shown, the system implements the blocks of the third method S300 described above to apply a first measurement specification and a second measurement specification different from the first measurement specification to the set of images, and then compile the results of the first and second measurement specifications into one graph or statistic.

[0457] In this implementation, the system can: receive a manual selection of a second specific feature in a first assembly unit represented in a first image in block S320; receive a selection of a second measurement type for the second specific feature in block S330; and extract a second true size of the second specific feature in the first assembly unit from the first image according to the second measurement type in block S344. In this example, the system can then for each image in the set of images: identify a second feature in the assembly unit represented in the image in block S340, where the second feature in the assembly unit is similar to the second specific feature in the first assembly unit; and extract a second true size of the second feature in the assembly unit from the image according to the second measurement type in block S344. In block S350, the system can then populate a two-dimensional graph with points representing a first measurement type across the set of images (e.g., along a first axis of a curve graph) and a second measurement type across the set of images (e.g., along a second axis of the curve graph). For example, the system can define the Cartesian coordinates of each image in the set as (result_first_measurement_specification, result_second_measurement_specification), and then represent each image in a two-dimensional scatter plot according to its Cartesian coordinates.

[0458] In this implementation, the system can also receive a mathematical model that links a first measurement specification to a second measurement specification, such as for an assembly unit in the set: a maximum and / or minimum difference between the results of the first measurement specification and the second measurement specification; a nominal sum and total tolerance of the sum of the results of the first measurement specification and the results of the second measurement specification; and so on. The system can then implement the above methods and techniques to automatically label assembly units that contain individual nominal sizes and dimensional tolerances that violate the first and second measurement specifications or features of a model that links the first and second measurement specifications.

[0459] 3.7.4 Filters

[0460] In one implementation, the system: receives a set of filtering values and a selection of a pre - existing measurement specification from a user; filters a set of images captured by an optical inspection station in an assembly line according to the filtering values; automatically applies the measurement specification, as described above, to the filtered set of images to generate a graphical curve plot and / or numerical representation of dimensions of features spanning a corresponding set of units; and passes the graphical curve plot and / or numerical representation to the user. For example, the user can search for or filter a set of pre - existing measurement specifications configured for assembly, configuration, or setup based on an assembly line identifier, optical inspection station sequence number, measurement specification origin or owner (e.g., the user who initially configured the measurement specification), measurement specification name, feature name or type, etc., and select a particular measurement specification (or a set of measurement specifications) to apply to a set of images captured by an optical inspection station inserted along the assembly line. In this example, the user can also input one or more unit filters, such as configuration (e.g., EVT, DVT, PVT), configuration (e.g., color, vendor, engineering), assembly date (or date range, time range), assembly stage, unit sequence number range, timestamp, optical inspection station or assembly status, fixture number, measurement value, measurement range, etc. The system can also store these filters, group the units based on any of the foregoing parameters, and generate a new plot corresponding to a set of images or a subset of images from a set of images based on one or more filters selected by the user.

[0461] In the foregoing example, the user can input a text natural - language filter into a prompt window in, for example, a local SMS messaging application executed on a smartphone, a local verification test application executed on a tablet, or a browser window on a desktop computer, and the system can implement natural - language processing techniques to convert the text string input by the user into a set of filters for the units and select a corresponding set of images and / or other data for the application of the measurement specification. Optionally, the system can publish a dynamic drop - down menu for filters that can be applied to a set of units previously imaged based on a measurement specification and / or other filters selected by the user. Then, as described above, the system can generate a graphical curve plot of the feature dimensions of the filtered set of units and push the graphical curve plot back to the user in the form of, for example, a static graphic visible in a local SMS text - messaging application executed on the user's smartphone or in the form of an interactive graphic visible in a local verification test application or in a browser window.

[0462] However, the system can interface with the user in any other way through a user interface executed at a local computing device to receive measurement specifications and / or assembly unit filters. The system can then: filter the set of images according to one or more filter values selected by the user to select a subset of images representing a subset of the assembly units; generate a graphical, textual, or statistical representation of similar features similar to a particular feature included in the subset of the assembly units; and then provide this graphical, textual, or statistical representation to the user via a user interface executed on the user's computing device or other local application or web browser.

[0463] 3.7.5 Notifications

[0464] In the above implementation of the foregoing methods and techniques executed by the system in (near) real time, the system can provide a prompt to inspect the marked assembly units to an electronic account associated with the user in real time. For example, the system can reproduce directly through the user interface an inspection prompt including the inspection prompt and the serial number of the marked assembly unit. For a system that remotely executes the blocks of the third method S300 from the user interface, the system can additionally or alternatively generate an SMS text message or an application-based notification including the inspection prompt and the serial number of the marked assembly unit, and then send the text message or notification to a mobile computing device (such as a smartphone, smartwatch) associated with the user in (near) real time, for example, when the user is in the building housing the assembly line or when the marked assembly unit is currently in production or currently housed. Optionally, for example, the system can asynchronously provide a graphical, textual, and / or statistical representation of the dimensions of similar features across the set of assembly units when the user activates the measurement specifications within the user interface or by sending a daily or weekly summary of the results of the measurement specifications to the user via email, as described below.

[0465] In a variation, the third method S300 may further include: receiving a subscription to the measurement specification from a user, for example, in the form of a request to receive an update related to the dimensions extracted from the image of the newly assembled unit according to the measurement space; and distributing a digital container to an electronic account associated with the user based on the subscription. In this variation, the system may enable the user and other users to subscribe to the measurement specification, and may automatically push, for example, the graphical and / or text prompts described above to the computing devices associated with each user who subscribes to the measurement specification. For example, the system may push an electronic notification to the mobile computing device associated with each user who subscribes to the measurement technique in real time to check a particular assembled unit that contains features with dimensions that are similar to a particular feature and exhibit a deviation from the nominal dimension defined for the particular feature or represent statistical outliers (i.e., not necessarily outside a predetermined tolerance). Optionally, the system may populate a spreadsheet with the serial numbers of the assembled units that contain such features exhibiting a deviation from the nominal dimension, insert the spreadsheet into the image, and send an email to the subscribers of the measurement specification, for example, in a daily or weekly summary.

[0466] 3.8 Image Review

[0467] The system may also execute the second method S200 in combination with the third method S300 to substantially align the images by the common features included in the assembled units represented in these images, and extract the dimensions of the common features or other similar features included in these assembled units. In this implementation, the system may then define the order of the images in the group based on the dimensions of the similar features extracted from the images in the group, and then scroll through the entire group of images in the user interface according to this order, enabling the user to view the images of the assembled units in the order of increasing (or decreasing) dimensions of the common features included in these assembled units and by which they are visually aligned. For example, the system may: define the order of the group of images based on the actual dimensions of the features similar to a particular feature extracted from the images in the group; substantially align the images in the group of images by features similar to a particular feature (e.g., by corner features or by two edge features) as described above; and in response to a scroll input at the user interface, reposition the images in the user interface in order throughout the reproduction of the images in the group.

[0468] In another implementation, the system can implement block S252 of the second method S200 in combination with block S350 of the third method S300 to select two (or more) images representing spans of dimensions of features similar to a particular feature and generate a composite image that includes these representative images. For example, in block S350, the system can: calculate a range of real dimensions spanning the set of real dimensions extracted from the set of images; select a second image representing a second assembly unit from the set of images, the second assembly unit including a second feature similar to the particular feature and characterized by a second dimension close to one end of this range of real dimensions; select a third image representing a third assembly unit from the set of images, the third assembly unit including a third feature similar to the particular feature and characterized by a third dimension close to the opposite end of the range of real dimensions; generate, as shown in block S252, a composite image including the second image and the third image overlaid on the second image; and then reproduce the composite image within the user interface. The system can then implement the above methods and techniques to combine two or more images as a whole based on the real dimensions of these features extracted from the set of images, the two or more images representing a subset of the assembly units including assembly units with similar features representing the set of assembly units.

[0469] The system and method can be at least partially embodied and / or implemented as a machine configured to receive a computer-readable medium storing computer-readable instructions. The instructions can be executed by computer-executable components integrated with hardware / firmware / software elements of an application, applet, host, server, network, website, communication service, communication interface, user computer or mobile device, a wristwatch, a smart phone, or any suitable combination thereof. Other systems and methods of the implementation can be at least partially embodied and / or implemented as a machine configured to receive a computer-readable medium storing computer-readable instructions. The instructions can be executed by computer-executable components integrated with computer-executable components integrated with devices and networks of the above types. The computer-readable medium can be stored on any suitable computer-readable medium, such as RAM, ROM, flash memory, EEPROM, optical devices (CD or DVD), hard disk drive, floppy disk drive, or any suitable device. The computer-executable components can be a processor, but any suitable dedicated hardware device can (optionally or additionally) execute the instructions.

[0470] As those skilled in the art will recognize from the previous detailed description, as well as from the drawings and the claims, modifications and variations can be made to the present invention without departing from the scope of the present invention as defined in the appended claims.

Claims

1. A method for automatically measuring common features across multiple assembly units, comprising: Displaying a first image from a set of images within a user interface, the form of the first image being recorded at an optical inspection station; Receiving a selection of a specific feature in a first assembly unit; Extracting a first true dimension of the specific feature in the first assembly unit from the first image, the first true dimension of a first measurement type selected from a set of measurement types including: Corner-to-corner distance; Edge length; Area; Radius; and Profile; Defining a feature window that encloses the specific feature, deviates from the specific feature, and is positioned relative to a first global origin of the first image; For each image in the set of images: Identifying a feature in the assembly unit represented in the image, the feature in the assembly unit being similar to the specific feature in the first assembly unit; and Extracting a true dimension of the feature in the assembly unit from the image, the true dimension of the first measurement type; and Aggregating the first true dimension and a set of true dimensions extracted from the set of images into a digital container.

2. The method according to claim 1, further comprising: Extracting a first set of features from the first image; Detecting the first global origin of the first image based on the first set of features; Receiving a selection of a measurement type for the specific feature; and For each image in the set of images: Extracting a set of features from the image; Detecting the global origin of the image based on the set of features; Positioning the feature window within the image relative to the global origin of the image; And Scanning a region of the image delimited by the feature window to identify the feature on the assembly unit represented in the image.

3. The method according to claim 1: Further comprising receiving a selection of a measurement type for the specific feature; Among them, Displaying the first image includes: Obtaining a first digital photographic image from a database, the first digital photographic image being recorded by the optical inspection station at a first time during assembly; Normalizing the first digital photographic image based on a reference image recorded at the optical inspection station to generate the first image; and Providing the first image to a computing device executing the user interface for reproduction; and Wherein, extracting the first true dimension of the specific feature in the first assembly unit from the first image includes: Projecting a dimension space onto the first image; and Extracting the first true dimension of the specific feature from the first image based on the position of the specific feature relative to the dimension space and the measurement type.

4. The method according to claim 3, further comprising: Obtaining a second digital photographic image from the database, the second digital photographic image being recorded by the optical inspection station at a second time during the assembly; Normalizing the second digital photographic image based on the reference image to generate a second image in the set of images; Identify a second feature in a second assembly unit represented in the second image, the second feature in the second assembly unit being similar to the specific feature in the first assembly unit; And Extracting a second true size of the second feature in the second assembly unit from the second image includes: Projecting the size space onto the second image; And Extracting a second true size of the second feature from the second image based on the position of the second feature relative to the size space and the measurement type.

5. The method according to claim 1, wherein Receiving a selection of the specific feature in the first assembly unit includes: Receiving a manual selection of a specific pixel from the first image; Identifying a first set of features in the first image; and Identifying the specific feature in the first set of features that is closest to the specific pixel.

6. The method according to claim 1, further comprising: Identifying an assembly state of the first assembly unit; And Selecting, from a body of images that are recorded across a set of optical inspection stations and represent multiple assembly units in various assembly states, a set of images that represent a set of assembly units in the assembly state.

7. The method according to claim 1, further comprising: Identifying a serial number of the first assembly unit; And Selecting, from a body of images that are recorded across a set of optical inspection stations and represent multiple assembly units in various assembly states, a set of images that represent the first assembly unit in various assembly stages.

8. The method according to claim 1, further comprising: Generating a measurement specification that defines the first measurement type in the set of measurement types and characterizes the specific feature; Receiving a subscription to the measurement specification from a user; And Distributing the digital container to an electronic account associated with the user based on the subscription.

9. The method according to claim 1, further comprising: Accessing a target size of the specific feature; Accessing a size tolerance of the target size of the specific feature; And Marking a second assembly unit in response to a second feature characterized by a second true size that differs from the target size by more than the size tolerance, the second feature being similar to the specific feature detected in a second image in a set of images of a second assembly unit in a set of assembly units.

10. The method according to claim 1, further comprising: Defining an order of the set of images based on true sizes of features similar to the specific feature extracted from images in the set of images; Substantially aligning the images in the set of images by features similar to the specific feature; And Transposing within the user interface in the order throughout the reproduction of the images in the set of images in response to a scroll input at the user interface.

11. The method according to claim 1: Among them, Aggregating the first true dimension and the set of true dimensions extracted from the set of images into the digital container includes aggregating the first true dimension and the set of true dimensions into a virtual histogram that includes a set of discrete percentile ranges spanning the first true dimension and the set of true dimensions; and The method further includes: reproducing the virtual histogram within the user interface; and in response to a selection of a particular percentile range within the set of discrete percentile ranges, reproducing within the user interface a particular image of the set of images that represents the particular percentile range.

12. The method according to claim 1, further comprising: accessing a dimension range of a feature similar to the particular feature associated with a failure of a component; identifying a second assembly unit that includes a second feature similar to the particular feature and characterized by a second true dimension included within the dimension range, the second assembly unit being represented in a second image of the set of images; and providing a prompt to inspect the second assembly unit to an electronic account associated with the user.

13. The method according to claim 1, wherein, Aggregating the first true dimension and the set of true dimensions extracted from the set of images into the digital container includes: calculating a true dimension range spanning the first true dimension and the set of true dimensions extracted from the set of images; selecting, within the set of images, a second image representing a second assembly unit that includes a second feature similar to the particular feature and characterized by a second dimension proximate a first end of the true dimension range; selecting, within the set of images, a third image representing a third assembly unit that includes a third feature similar to the particular feature and characterized by a third dimension proximate a second end of the true dimension range; generating a composite image that includes the second image and the third image overlaid on the second image; and reproducing the composite image within the user interface.

14. The method according to claim 5: Among them, Displaying the first image within the user interface includes: generating a first feature space that includes a first set of vectors representing the first set of features; and displaying the first feature space within the user interface together with the first image; wherein receiving a selection of the particular feature within the first assembly unit includes: receiving a manual selection of a particular vector feature from the first set of vectors included within the first feature space; and identifying the particular feature corresponding to the particular vector.

15. The method according to claim 14, wherein, Identifying a feature within an assembly unit represented in an image for each image of the set of images includes, for each image of the set of images: generating a feature space that includes a set of vectors representing the set of features; aligning the feature space with the first feature space based on a global origin; scanning a region of the image delimited by the feature window to identify a vector within the set of vectors that is closest in position and geometry to the particular vector of the first set of vectors; and mark the features in the image corresponding to the vector as similar to the specific feature.

16. A method, comprising: obtaining a set of images; for a first image in the set of images: displaying the first image within a user interface, the form of the first image being recorded at an optical inspection station; identifying a first set of features in the first image; receiving a selection of a specific feature in a first assembly unit from the first set of features; extracting from the first image a first true dimension associated with the specific feature in the first assembly unit, the first true dimension being of a first measurement type selected from a set of measurement types, including: corner-to-corner distance; edge length; area; radius; and contour; and defining a feature window that encloses the specific feature, deviates from the specific feature, and is positioned relative to a first global origin of the first image; for a second image in the set of images: positioning the feature window within the second image relative to a second global origin of the second image; identifying a second feature in a second assembly unit represented in the second image, the second feature in the second assembly unit being similar to the specific feature in the first assembly unit; and extracting from the second image a second true dimension associated with the second feature in the second assembly unit, the second true dimension being of the first measurement type; for a third image in the set of images: positioning the feature window within the third image according to a third global origin of the third image; and identifying a third feature in a third assembly unit represented in the third image, the third feature in the third assembly unit being similar to the specific feature in the first assembly unit; and extracting from the third image a third true dimension associated with the third feature in the third assembly unit, the third true dimension being of the first measurement type; in response to a selection of the second image at the user interface, displaying the second image and the second true dimension within the user interface; and in response to a selection of the third image at the user interface, displaying the third image and the third true dimension within the user interface.

17. The method according to claim 16, further comprising: for the first image in the set of images: extracting a first set of features from the first image; detecting the first global origin of the first image based on the first set of features; determining the measurement type of the specific feature; extracting from the first image the first true dimension associated with the specific feature in the first assembly unit according to the measurement type; displaying the first true dimension within the user interface together with the first image; for the second image in the set of images: extracting a second set of features from the second image; detecting the second global origin of the second image based on the second set of features; scanning a region of the second image delimited by the feature window to identify the second feature in the second assembly unit represented in the second image; and Extract the second true dimension related to the second feature in the second assembly unit from the second image according to the measurement type; and For the third image in the set of images: Extract a third set of features from the third image; Based on the third set of features, detect the third global origin of the third image; Scan the region of the third image delimited by the feature window to identify the third feature in the third assembly unit represented in the third image; and Extract the third true dimension related to the third feature in the third assembly unit from the third image according to the measurement type.

18. The method according to claim 17: Among them, Scanning the region of the second image delimited by the feature window to identify the second feature in the second assembly unit represented in the second image includes performing a feature selection routine to identify the second feature in the second image; The method further includes, in response to identifying the second feature in the second assembly unit represented in the second image: Display the second image within the user interface; and Indicate the second feature within the second image; and wherein scanning the region of the third image delimited by the feature window to identify the third feature in the third assembly unit represented in the third image includes, in response to receiving a manual confirmation that the second feature is similar to the specific feature at the user interface, identifying the third feature in the third image according to the feature selection routine.

19. The method according to claim 17: Among them, Receiving the selection of the specific feature in the first assembly unit includes receiving the selection of the specific feature in the first assembly unit at a first time; wherein identifying the second feature in the second assembly unit represented in the second image includes identifying the second feature in the second assembly unit represented in the second image recorded at the assembly line before the first time; and The method further includes: Access the target dimension of the specific feature; Access the dimensional tolerance of the target dimension of the specific feature; In response to receiving a fourth image recorded at a second time after the first time, scan the region of the image delimited by the feature window to identify a fourth feature in a fourth assembly unit represented in the fourth image, the fourth feature in the fourth assembly unit being similar to the specific feature in the first assembly unit; Extract the fourth true dimension of the fourth feature in the fourth assembly unit from the fourth image according to the measurement type; and In response to the fourth true dimension differing from the target dimension by more than the dimensional tolerance, mark the fourth assembly unit.

20. A method for automatically measuring common features across multiple assembly units, comprising: Display a first image in a set of images within a user interface, the first image being in a form recorded at an optical inspection station; Receive a selection of a specific feature in a first assembly unit represented in the first image, wherein a measurement type of the specific feature includes a set of measurement types, and the set of measurement types includes: corner-to-corner distance, edge length, area, radius, and profile; Extract a first true size of the specific feature in the first assembly unit from the first image, and the first true size of a first measurement type selected from the set of measurement types includes: Corner-to-corner distance; Edge length; Area; Radius; and Profile; For each image in the set of images: Identify a feature in an assembly unit represented in the image, the feature in the assembly unit being similar to the specific feature in the first assembly unit; and Extract a true size of the feature in the assembly unit from the image, the true size of the first measurement type; and Aggregate the first true size and a set of true sizes extracted from the set of images into a digital container; and Mark the second assembly unit in response to a second feature characterized by a second true size detected in a second image of the set of images of a second assembly unit in the set of assembly units deviating from the set of true sizes.

Images