System and method for displacement and strain detection using color tracking

Abstract

Optical inspection systems, methods, and media are configured to perform and / or implement, for a plurality of image frames of a target object captured by an image sensor, the target object including at least one fiducial marker: receiving a respective image frame, the image frame including an n-dimensional array of pixel values corresponding to a plurality of colors, converting the image frame to a color-adjusted image, the color-adjusted image including a two-dimensional array of pixel values corresponding to a first color of the plurality of colors, and determining at least one centroid corresponding to at least one region in the color-adjusted image tracking the at least one centroid across at least two of the plurality of image frames; and outputting an indication of a mechanical parameter of the target object based on the tracking.

Description

SYSTEM AND METHOD FOR DISPLACEMENT AND STRAIN DETECTION USINGCOLOR TRACKINGCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to and the benefit of U.S. Provisional Application No. 63 / 519,144, filed August 11, 2023 and titled “System and Method for Accurate Displacement and Strain Measurement Using a Color Tracking Algorithm,” the entire contents of which are herein incorporated by reference for all purposes.STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

[0002] This invention was made with government support under DE-EE0009239 awarded by the Department of Energy. The government has certain rights in the invention.TECHNICAL FIELD

[0003] This disclosure relates generally to optical inspection and, in particular, to systems and methods for determining displacement, strain, or other mechanical parameters in a non-contact manner using color tracking.BACKGROUND

[0004] Accurate measurement of displacement and strain in materials and structures is important for understanding their mechanical behavior, characterizing their properties, and ensuring the reliability and safety of engineering structures. Comparative methods such as strain gauges and extensometers have been employed historically, but they generally come with limitations such as the need for direct contact with the material, expense, or the inability to capture localized strains accurately.

[0005] Optical techniques have been proposed to address some of these problems due to their non-contact nature, high spatial resolution, and ability to capture full-field deformations. Digital image correlation (DIC) is one comparative optical technique, which involves tracking speckle patterns surface of a material to determine the displacement and strain. However, DICtypically requires expensive hardware and software and can be time-consuming to implement. Moreover, in cases where comparative methods, such as DIC, are impractical due to the inability to mark the material with a speckle pattern (e.g., because the material under inspection is very thin, such as a carbon fiber) or too expensive, alternative techniques are required.

[0006] Color tracking is a technique that allows for the detection of a specific color or color pattern in an image. This technique has been used in a range of applications such as bridge deflections, tracking fill dirt, and target tracking. Color tracking can be performed using standard imaging equipment making it a more cost-effective as a potential alternative is to use a smartphone camera. However, comparative techniques using color tracking have been unable to provide accurate modeling. For example, color tracking modalities using edge tracking algorithms have provided poor accuracy in general cases, and even poorer accuracy in cases where strain or stress acts on the feature under inspection. Moreover, comparative examples of color tracking have been unable to provide measurements of strain. Therefore, there is a need for alternative techniques that offer improved accuracy, flexibility, and cost-effectiveness.SUMMARY

[0007] The present disclosure addresses needs in the field by presenting systems, methods, and computer readable media for providing, among other things, a) accurate displacement and strain measurements using a cost-effective color camera and color tracking algorithm and / or b) accurate displacement and strain measurement using a color tracking algorithm.

[0008] Although example embodiments of the present disclosure are explained in some instances in detail herein, it is to be understood that other embodiments are contemplated. Accordingly, it is not intended that the present disclosure be limited in its scope to the details of construction and arrangement of components set forth in the following description or illustrated in the drawings. The present disclosure is capable of other embodiments and of being practiced or carried out in various ways.

[0009] According to one aspect of the present disclosure, an optical inspection method is provided. The method comprises, for a plurality of image frames of a target object captured by an image sensor, the target object including at least one fiducial marker: receiving a respective image frame, the image frame including an n-dimensional array of pixel values corresponding to aplurality of colors, converting the image frame to a color-adjusted image, the color-adjusted image including a two-dimensional array of pixel values corresponding to a first color of the plurality of colors, and determining at least one centroid corresponding to at least one region in the color- adjusted image tracking the at least one centroid across at least two of the plurality of image frames; and outputting an indication of a mechanical parameter of the target object based on the tracking.

[0010] According to another aspect of the present disclosure, an optical inspection system is provided. The system comprises an imaging system configured to capture a plurality of image frames of a target object, the target object including at least one fiducial marker; and at least one processor configured to, for the plurality of image frames: receive a respective image frame, the image frame including an n-dimensional array of pixel values corresponding to a plurality of colors, convert the image frame to a color-adjusted image, the color-adjusted image including a two-dimensional array of pixel values corresponding to a first color of the plurality of colors, and determine at least one centroid corresponding to at least one region in the color-adjusted image, track the at least one centroid across at least two of the plurality of image frames, and output an indication of a mechanical parameter of the target object based on the tracking.

[0011] According to another aspect of the present disclosure, a non-transitory computer- readable medium is provided. The non-transitory computer-readable medium stores instructions that, when executed by at least one processor of a computer, cause the computer to perform operations comprising, for a plurality of image frames of a target object captured by an image sensor, the target object including at least one fiducial marker: receiving a respective image frame, the image frame including an n-dimensional array of pixel values corresponding to a plurality of colors, converting the image frame to a color-adjusted image, the color-adjusted image including a two-dimensional array of pixel values corresponding to a first color of the plurality of colors, and determining at least one centroid corresponding to at least one region in the color-adjusted image tracking the at least one centroid across at least two of the plurality of image frames; and outputting an indication of a mechanical parameter of the target object based on the tracking.BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Various objects, features, and advantages of the disclosed subject matter can be more fully appreciated with reference to the following detailed description of the disclosed subjectmatter when considered in connection with the following drawings, in which like reference numerals identify like elements.

[0013] FIG. 1 shows an example of the output of a color tracking algorithm in accordance with various aspects of the present disclosure.

[0014] FIG. 2 shows an example of the effects of removal factor and threshold variables in accordance with various aspects of the present disclosure.

[0015] FIG. 3 shows a set of example graphs from a displacement resolution experiment in accordance with various aspects of the present disclosure.

[0016] FIG. 4 shows an example of displacement measurement test results in accordance with various aspects of the present disclosure.

[0017] FIG. 5 shows an example of tensile test results in accordance with various aspects of the present disclosure.

[0018] FIG. 6 shows an example of arm flexion test results in accordance with various aspects of the present disclosure.

[0019] FIG. 7 shows an example schematic of a machine on which various aspects of the present disclosure can be implemented.

[0020] FIG. 8 shows an example of an optical inspection method in accordance with various aspects of the present disclosure.DETAILED DESCRIPTION

[0021] The detailed description set forth below in connection with the appended drawings is intended as a description of various configurations and is not intended to represent the only configurations in which the subject matter described herein may be practiced. The detailed description includes specific details to provide a thorough understanding of various aspects of the present disclosure. However, it will be apparent to those skilled in the art that the various features, concepts, and aspects described herein may be implemented and practiced without these specific details.

[0022] It should be appreciated that any element, part, section, subsection, or component described with reference to any specific embodiment above may be incorporated with, integrated into, or otherwise adapted for use with any other embodiment described herein unless specifically noted otherwise or if it should render the embodiment device non-functional. Likewise, any stepdescribed with reference to a particular method or process may be integrated, incorporated, or otherwise combined with other methods or processes described herein unless specifically stated otherwise or if it should render the embodiment method nonfunctional. Furthermore, multiple embodiment devices or embodiment methods may be combined, incorporated, or otherwise integrated into one another to construct or develop further embodiments of the invention described herein.

[0023] It should be appreciated that any of the components or modules referred to with regards to any of the present invention embodiments discussed herein, may be integrally or separately formed with one another. Further, redundant functions or structures of the components or modules may be implemented. Moreover, the various components may be communicated locally and / or remotely with any user / operator / customer / client or with any machine / system / computer / processor. Moreover, the various components may be in communication via wireless and / or hardwire or other desirable and available communication means, systems, and hardware. Moreover, various components and modules may be substituted with other modules or components that provide similar functions.

[0024] It should be appreciated that the device and related components discussed herein may take on all shapes along the entire continual geometric spectrum of manipulation of x, y, and z planes to provide and meet the environmental, anatomical, and structural demands and operational requirements. Moreover, locations and alignments of the various components may vary as desired or required.

[0025] It should be appreciated that various sizes, dimensions, contours, rigidity, shapes, flexibility, and materials of any of the components or portions of components in the various embodiments discussed throughout may be varied and utilized as desired or required.

[0026] It should be appreciated that while some dimensions are provided on the aforementioned figures, the device may constitute various sizes, dimensions, contours, rigidity, shapes, flexibility and materials as it pertains to the components or portions of components of the device, and therefore may be varied and utilized as desired or required.

[0027] It must also be noted that, as used in the specification and the appended claims, the singular forms “a,” “an” and “the” include plural referents unless the context clearly dictates otherwise. Ranges may be expressed herein as from “about” or “approximately” one particularvalue and / or to “about” or “approximately” another particular value. When such a range is expressed, other exemplary embodiments include from the one particular value and / or to the other particular value.

[0028] By “comprising” or “containing” or “including” is meant that at least the named compound, element, particle, or method step is present in the composition or article or method, but does not exclude the presence of other compounds, materials, particles, or method steps, even if the other such compounds, material, particles, or method steps have the same function as what is named.

[0029] In describing example embodiments, terminology will be resorted to for the sake of clarity. It is intended that each term contemplates its broadest meaning as understood by those skilled in the art and includes all technical equivalents that operate in a similar manner to accomplish a similar purpose. It is also to be understood that the mention of one or more steps of a method does not preclude the presence of additional method steps or intervening method steps between those steps expressly identified. Steps of a method may be performed in a different order than those described herein without departing from the scope of the present disclosure. Similarly, it is also to be understood that the mention of one or more components in a device or system does not preclude the presence of additional components or intervening components between those components expressly identified.

[0030] Some references, which may include various patents, patent applications, and publications, are cited in a reference list and discussed in the disclosure provided herein. The citation and / or discussion of such references is provided merely to clarify the description of the present disclosure and is not an admission that any such reference is “prior art” to any aspects of the present disclosure described herein. For example, the devices, systems, apparatuses, modules, compositions, articles of manufacture, materials, computer program products, non-transitory computer readable medium, and methods of various embodiments of the invention disclosed herein may utilize aspects (such as devices, apparatuses, modules, systems, compositions, articles of manufacture, materials, computer program products, non-transitory computer readable medium, and methods) disclosed in the following references, applications, publications and patents and which are hereby incorporated by reference herein in their entirety (and which are not admitted to be prior art with respect to the present invention by inclusion in this section): A. U. S. Utility Patent Application Serial No. 18 / 273,658, entitled “METHODS AND SYSTEMS FOR PRODUCINGDISPERSED GRAPHENE FROM SPENT LITHIUM-ION BATTERIES”, filed July 21, 2023; B. International Patent Application Serial No. PCT / US2022 / 013542, entitled “METHODS AND SYSTEMS FOR PRODUCING DISPERSED GRAPHENE FROM SPENT LITHIUM-ION BATTERIES”, filed January 24, 2022; Publication No. WO 2022 / 177701, August 25, 2022; C. International Patent Application Serial No. PCT / US2023 / 068927, entitled “Hydrogen Passivation Aided Conversion of Cotton to High-Quality Graphene”, filed June 23, 2023; D. U.S. Utility Patent Application Serial No. 17 / 617,192, entitled “SYSTEM AND METHOD OF ACCELERATING POLYMER FIBER STABILIZATION VIA IRRADIATION TREATMENT”, filed December 07, 2021; Publication No. US 2022-0235516 Al, July 28, 2022; E. International Patent Application Serial No. PCT / US2020 / 037010, entitled “SYSTEM AND METHOD OF ACCELERATING POLYMER FIBER STABILIZATION VIA IRRADIATION TREATMENT”, filed June 10, 2020; Publication No. WO 2020 / 252031, December 17, 2020; F. U.S. Utility Patent Application Serial No. 15 / 753,866, entitled “METHOD OF FORMING GRAPHENE / METAL-OXIDE HYBRID REINFORCED COMPOSITES AND PRODUCT THEREOF”, filed February 20, 2018; U.S. Patent No. 10,781,142, issued September 22, 2020; G. International Patent Application Serial No. PCT / US2016 / 047714, entitled “METHOD OF FORMING GRAPHENE / METAL- OXIDE HYBRID REINFORCED COMPOSITES AND PRODUCT THEREOF”, filed August 19, 2016; Publication No. WO2017031403, February 23, 2017; and H. U.S. Utility Patent Application Serial No. 15 / 274,557, entitled “PROCESS OF FORMING ELECTRODES AND PRODUCTS THEREOF FROM BIOMASS”, filed September 23, 2016; U.S. Patent No. 10,446,329, issued October 15, 2019.

[0031] It should be appreciated that as discussed herein, a subject may be a human or any animal. It should be appreciated that an animal may be a variety of any applicable type, including, but not limited thereto, mammal, veterinarian animal, livestock animal or pet type animal, etc. As an example, the animal may be a laboratory animal specifically selected to have certain characteristics similar to human (e.g. rat, dog, pig, monkey), etc. It should be appreciated that the subject may be any applicable human patient, for example.

[0032] The term “about,” as used herein, means approximately, in the region of, roughly, or around. When the term “about” is used in conjunction with a numerical range, it modifies that range by extending the boundaries above and below the numerical values set forth. In general, the term “about” is used herein to modify a numerical value above and below the stated value by avariance of 10%. Tn one aspect, the term “about” means plus or minus 10% of the numerical value of the number with which it is being used. Therefore, about 50% means in the range of 45%-55%. Numerical ranges recited herein by endpoints include all numbers and fractions subsumed within that range (e.g. 1 to 5 includes 1, 1.5, 2, 2.75, 3, 3.90, 4, 4.24, 5, and so on). Similarly, numerical ranges recited herein by endpoints include subranges subsumed within that range (e g. 1 to 5 includes 1-1.5, 1.5-2, 2-2.75, 2.75-3, 3-3.90, 3.90-4, 4-4.24, 4.24-5, 2-5, 3-5, 1-4, 2-4, and so on). It is also to be understood that all numbers and fractions thereof are presumed to be modified by the term “about.”

[0033] It is also to be understood that any reference to an element herein using a designation such as “first,” “second,” and so forth does not limit the quantity or order of those elements, unless such limitation is explicitly stated. Rather, these designations may be used herein as a convenient method of distinguishing between two or more elements or instances of an element. Thus, a reference to first and second elements does not mean that only two elements may be employed or that the first element must precede the second element in some manner.

[0034] Also as used herein, unless otherwise limited or defined, “or” indicates a nonexclusive list of components or operations that can be present in any variety of combinations, rather than an exclusive list of components that can be present only as alternatives to each other. For example, a list of “A, B, or C” indicates options of: A; B; C; A and B; A and C; B and C; and A, B, and C. Correspondingly, the term “or” as used herein is intended to indicate exclusive alternatives only when preceded by terms of exclusivity, such as, e.g., “either,” “one of,” “only one of,” or “exactly one of.” Further, a list preceded by “one or more” (and variations thereon) and including “or” to separate listed elements indicates options of one or more of any or all of the listed elements. For example, the phrases “one or more of A, B, or C” and “at least one of A, B, or C” indicate options of: one or more A; one or more B; one or more C; one or more A and one or more B; one or more B and one or more C; one or more A and one or more C; and one or more of each of A, B, and C. Similarly, a list preceded by “a plurality of’ (and variations thereon) and including “or” to separate listed elements indicates options of multiple instances of any or all of the listed elements. For example, the phrases “a plurality of A, B, or C” and “two or more of A, B, or C” indicate options of: A and B; B and C; A and C; and A, B, and C. In general, the term “or” as used herein only indicates exclusive alternatives (e.g., “one or the other but not both”) when preceded by terms of exclusivity, such as, e.g., “either,” “one of,” “only one of,” or “exactly one of.”

[0035] The present disclosure includes a description of various methods. For any methods described, regardless of whether the method is described in conjunction with a flow diagram, it should be understood that unless otherwise specified or required by context, any explicit or implicit ordering of steps performed in the execution of a method does not necessarily imply that those steps must be performed in the order presented, but instead the steps may be performed in a different order and / or in parallel.

[0036] The present disclosure may be implemented on or with the use of computing devices including control units, processors, and / or memory elements in some examples. As used herein, a “control unit” may be any computing device configured to send and / or receive information (e.g., including instructions) to / from various systems and / or devices. A control unit may comprise processing circuitry configured to execute operating routine(s) stored in a memory. The control unit may comprise, for example, a processor, microcontroller, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), and the like, any other digital and / or analog components, as well as combinations of the foregoing, and may further comprise inputs and outputs for processing control instructions, control signals, drive signals, power signals, sensor signals, and the like. All such computing devices and environments are intended to fall within the meaning of the term “controller,” “control unit,” “processor,” or “processing circuitry” as used herein unless a different meaning is explicitly provided or otherwise clear from the context. The term “control unit” is not limited to a single device with a single processor, but may encompass multiple devices (e.g., computers) linked in a system, devices with multiple processors, special purpose devices, devices with various peripherals and input and output devices, software acting as a computer or server, and combinations of the above. In some implementations, the control unit may be configured to implement cloud processing, for example by invoking a remote processor.

[0037] Moreover, as used herein, the term “processor” may include one or more individual electronic processors, each of which may include one or more processing cores, and / or one or more programmable hardware elements. The processor may be or include any type of electronic processing device, including but not limited to central processing units (CPUs), graphics processing units (GPUs), ASICs, FPGAs, microcontrollers, digital signal processors (DSPs), or other devices capable of executing software instructions. When a device is referred to as “including a processor,” one or all of the individual electronic processors may be external to the device (e.g., to implement cloud or distributed computing). In implementations where a device has multipleprocessors and / or multiple processing cores, individual operations described herein may be performed by any one or more of the microprocessors or processing cores, in series or parallel, in any combination.

[0038] As used herein, the term “memory” may be any storage medium, including a nonvolatile medium, e.g., a magnetic media or hard disk, optical storage, or flash memory, including read-only memory (ROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM); a volatile medium, such as system memory, e.g., random access memory (RAM) such as dynamic RAM (DRAM), synchronous dynamic RAM (SDRAM), static RAM (SRAM), extended data out (EDO) DRAM, extreme data rate dynamic (XDR) RAM, double data rate (DDR) SDRAM, etc.; on-chip memory; and / or an installation medium where appropriate, such as software media, e.g., a CD-ROM, a DVD-ROM, a Blu-ray disc, or floppy disks, on which programs may be stored and / or data communications may be buffered. The term “memory” may also include other types of memory or combinations thereof. For the avoidance of doubt, cloud storage is contemplated in the definition of memory.

[0039] Additional descriptions of aspects of the present disclosure will now be provided with reference to the accompanying drawings. The drawings form a part hereof and show, by way of illustration, specific embodiments or examples.

[0040] The present disclosure presents the application of a color tracking algorithm that can be used with any camera to determine the displacement and strain of materials and structures using a fiducial marker. The fiducial may be, for example, a sticker applied to the target object, a three-dimensional bead adhered to the target object, a paint or other marker applied to the target object, and the like. The fiducial may have any shape, including circular, square, rectangular, polygonal, spherical, cuboidal, polyhedral, etc. In an example, a mobile phone camera may be used as the camera. The algorithm set forth herein calculates the displacements and strain by finding the centroid of the specified color and is compared to the results obtained from DIC correlation analysis as proof-of-concept. The findings reveal that the color tracking algorithm was able to capture the displacement and strain of the fiducial marker with higher precision and lower noise compared to using DIC as a point tracking measurement. This technique may be extended to stereo measurements, and as such the color tracking approach presents use cases for numerous applications in cost-effective manner. The ability to utilize a mobile phone camera allows for more accessibility to this technique and the use of fiducial marker (e.g., a green marker) is simple andeasy to implement. This provides possibilities for widespread adoption in various fields where precise displacement and strain measurements are required and as an educational tool for students learning about optical techniques. In addition to the displacement and strain example, an example of this technique is used to measure the movement of a human arm. However, these experiments are presented merely by way of example. In practice, the systems and methods set forth herein may be applied to any tensile or compressive test.

[0041] Experimental Materials, Methods, and Results

[0042] The present disclosure includes a discussion directed to enabling the use of a color tracking algorithm to determine the displacement and strain of materials through experimental proofs-of-concept. Three different experiments were used to assess the tracking ability. The first experiment was a nylon fiber with a green ball attached to the base. The color tracking algorithm was compared to the results of a DIC correlation of the fiducial marker. The color tacking algorithm was able to capture more of the movement of the ball with less noise than the DIC correlation. The second experiment was a carbon fiber tow tensile test. The experiment included two fiducial markers allowing for the calculation of the strains during loading. The third experiment was a biomechanics application visualizing the movement of a human arm. The experiment included three fiducial markers and was able to measure the displacement of the markers and the angle of the joint.

[0043] An image captured from a camera is outputted to a 3D matrix with red, green, and blue color intensity values. Color tracking algorithms are a particular branch of computer vision that can be used to detect and track specific colors or color patterns in an image or video. The first step in a color tracking algorithm is to identify the colors or color patterns that need to be tracked. The color tracking algorithm used for this example combines the colors from the different channels to determine an intensity in regard to a specific color. For this case, green was utilized because color cameras have two times the amount of green filters compared to red and blue. The intensity of the green intensity was calculated using the Equation (1). It is implemented on each pixel to convert the 3D matrix to a 2D array of green intensity values.I = g — ar — ab (1)

[0044] In Equation (1), r, g, and b are the intensity values from the red, green, and blue channels, respectively, and a is a removal factor. In implementations where red or blue are utilizedinstead of green, Equation (1) may be modified accordingly. In any case, the intensity values are normalized to better assess the results of the intensity calculation. The normalization is performed using Equation (2).

[0045] FIG. 1 shows the colors that are found after performing the color tracking algorithm. In FIG. 1, images (a) and (b) show the colors for the full range of RGB values at the white and dark intensity values, respectively. Image (c) shows the unwrapped version of the colors on the maximum surface of the RGB values, and images (d) through (I) show the resulting colors remaining after running the color tracking algorithm for different values of the removal factor a namely, 0.2, 0.5, and 0.8. As can be seen from FIG. 1, careful selection of a can lead to a down selection of colors detected in the algorithm.

[0046] The color tracking experiments were captured using a Google Pixel 2 XL. The smartphone is equipped with 2 cameras: a main back facing camera and secondary front facing camera. The back facing camera is a 12.2 MP, f / 1.8, 27 mm wide, 1 / 2.55", 1.4 pm sensor size, dual pixel PDAF, Laser AF, OIS camera and is the only one used for this example. When in video mode the camera specification changes to 4K at 30 frames per second (fps), 1080p at 30 / 60 / 120 fps, or 720p at 240 fps. Each experiment discussed here was captured in video mode carrying 30 fps to 120 fps. The captured videos were processed with the phone’s native software. The resulting video files were then transferred to a computer for further analysis.

[0047] A custom script was created in MATLAB 2022b to perform the color tracking. The script starts by importing the video and extracting an image from the desired frame. The image is then separated into the red, green, and blue channels. The intensities of the green values are calculated via Equation (1) above. To determine the appropriate removal factor, a, and threshold value, T, a small parametric study was performed to assess the effect as shown in FIG. 2. In FIG. 2, the rows of the image matrix correspond to T = 0.1, 0.4, 0.6, and 0.8 from top to bottom, and the columns of the image matrix correspond to a = 0.1, 0.4, 0.6, and 0.8 from left to right. Thus, the image matrix shows the results of changing the removal factor and threshold value.

[0048] Increasing the removal factor and threshold closer to 1 allows for a more defined selection. Using too small of a value for a (columns 1 one and two) results in too many pixelsbeing selected and too high of a value for a (column four) results in less of the object identified. The value of a for this example was set to 0.6 based on these results and not changed. The lowest value of T of 0.1 (row one) includes features that are not anticipated even with the highest value of a. Increasing the value of T to 0.4 (row two) more clearly shows the effect of the value of a. As the value of a increases, the selection comes closer to only selecting the green dot from the image. Further increasing the value of T (row three) makes the selection more precise. However, increasing the value of T too far (row four) leaves less of the green marker in the image and therefore provides fewer pixels to find the centroid.

[0049] By selecting particular values of the removal factor a and the threshold value T, only the green dot remains. The particular values may be manually selected or automatically determined. The script then fines the centroid of the dots by using the ‘regionprops’ function implemented in MATLAB. This function finds so-called regions from a binary image and determines the area and centroid of that region. The region results are sorted from largest to smallest. The first entries are selected typically corresponding to the number of dots used within the experiment. For one dot, there is no further refinement needed and the centroid of the data is found for the dot in all the frames. For multiple dots, the full region is sorted by area and down selected to only the number of dots in the images. Using multiple dots can allow for more complex measurements such as strain or angles.

[0050] For strain measurements, the initial frame (or multiple zero-strain frames) is used to calculate an original length. Afterwards, the change in length is calculated for each frame and divided by the original length to calculate strain data.

[0051] The displacement resolution was determined by taking a video of one fiducial marker on a stationary wall. A video was taken from the camera for ~1 minute at 30 fps capturing 1800 images. In addition, the stand-off distance was varied from 10 to 250 cm to determine the change as the field of view (FOV) became larger. The images were then processed using the algorithm of the present disclosure to determine the centroid of the fiducial marker. In some practical implementations, an algorithm may analyze an image taken at one stand-off distance and, based on the analysis (e.g., based on a signal to noise ratio, based on a determination of whether the fiducials are properly in frame), instruct an operator of the camera to move closer to or further from the target object. In other examples, the operator may instead manually verify that the standoff distance is appropriate.

[0052] FIG. 3 shows a series of graphs from the 10 cm stand-off distance displacement resolution experiment. Graph (a) shows a photograph of the full image, and graph (b) shows an image zoomed in toward the centroid value with experimental measurements overlaid on the first frame. In graphs (a) and (b), Lfovis the diagonal length of the FOV, Cavgis the average position of the centroid, Exp is the experimental measurements, and Dmaxis the maximum distance from the average centroid. The blue circles in graph (b) are all the centroids extracted from the experiment, and the dashed line shows a circle that exhibits the maximum distance from the average centroid. From graph (b), it can be seen that the method set forth herein has subpixel resolution.

[0053] The displacement strain results are shown in graph (c) for the average noise as a function of stand-off distance, and in graph (d) for relative noise. The pixel noise is determined by taking the average distance from the centroid and is shown as both physical and pixel units. As can be seen from graph (c), the average pixel noise grows as stand-off distance increases, but then drops. This may be due to less pixels being available for the centroid to generate noise. However, the physical distance shows that even though the noise is lower in pixel units, the physical units increase as stand-off distance increases. Graph (d) shows the average noise normalized by the length of the FOV Lfov. The relative noise shows the same trend as the average noise.

[0054] Because these are point measurements, the strain s is related to the gauge length according to Equation (3).8 = Y (3)

[0055] In Equation (3), AL is the change in length, and L is the original gauge length. Applying the noise to the system and assuming that the change in length in zero, Equation (3) becomes Equation (4).

[0056] In Equation (4), N is the noise level. As the gauge length decreases, the noise increases, thus making the strain resolution variable.

[0057] A moving pendulum experiment was conducted as an example of displacement tracking with the higher frame rate (120 fps). A green bead was strung through a tow of nylon fibers that was tied to a weight. The whole system is rigidly moving in an upwards direction untilout of the frame. A tow of nylon fibers was tied to an S-hook to act as a weight. A green bead was threaded through the fibers and sat atop of the weight. The fibers were then clamped to an Admet expert universal tensile machine (UTM). The weight was moved and released to create a pendulum. The Admet UTM was then jogged in an upward direction at a rate of 100 mm / min. The camera captured the movement of the ball until it proceeded out of frame. As a form of comparison, tracking of the point was performed with a DIC process using VIC -2D software. The full image was used as an area of interest. The subsets were varied to contain the most data over the time and provide a compromise between noise and accuracy.

[0058] FIG. 4 presents the horizontal and vertical displacements from the proposed color tracking (CT) algorithm, and the DIC correlation. Image (a) shows the full displacement time history of both measurements overlay ed on the final frame of the video. It shows that the DIC and CT measurements were close over the first -10% of the video. However, the DIC algorithm decorrelated after this point, as can be seen from graphs (b) and (d). The failure of DIC is due to the lack of a random speckle pattern on this test and the fact that DIC is developed for small displacements and small strains. Graphs (c) and (e) show the region that the DIC correlated. Over this region, the DIC correlated well and maintained similar measurements with the color tracking algorithm within 5%. However, it was noisier than the color tracking especially near the transitions or the peaks of the wave. The average of the absolute difference between these two sets of data are 3.08 and 1.14 pixels for the horizontal and vertical displacements, respectively.

[0059] A tensile test was conducted on a carbon fiber tow specimen to evaluate the performance of the color tracking algorithm in determining strain between two fiducial markers. The ability to track the displacement on the material rather than the grips is important to truly testing material properties due to inaccuracies with the measurements. These inaccuracies stem from errors such as compliance of the test machine with stiff materials, grip slippage, and the like. The tow was made ofZoltek Panex 35 carbon fiber with West Systems 105 epoxy resin with -55% volume fraction. The tensile test was performed in accordance with ASTM D4018. The test was recorded at 120 fps frame rate for 35 seconds. FIG. 5 illustrates the tensile test. Image (a) shows the experimental setup. Two fiducial markers, green stickers, were placed on the surface of the cured tow. The specimen was subjected to a uniaxial tensile load using an Admet expert UTM 2600, and the displacement and strain were recorded using the color tracking algorithm. Graph (b) shows the stress-strain results. The stress-strain relationship, as expected, was linear. Thecalculated modulus closely matched the values reported in the Zoltek material datasheet. This demonstrated the ability for the algorithm to accurately capture the strain and displacement even for small displacements.

[0060] The second experiment was an application to test the arm flexion. Attached to a human arm are three dots. A video was captured to monitor the movement of the three dots. The subject was requested to bring the arm from his side up towards his face. This was achieved by collapsing the elbow joint and bringing up the hand, as seen in images (a)-(c), which were taken at t = 0 s, 0.46 s, and 0.87 s, respectively. The motion was repeated three more times. Graphs (d) and (e) show the displacements of point 1 and 2 in the horizontal and vertical directions, respectively, and the angle joint 1 makes between the two linkages. As can be seen, the tracking is able to accurately capture the movement of the arm during this experiment.

[0061] The accurate determination of displacement and strain was demonstrated through the use of a color tracking algorithm with the use of three different experiments. The close agreement between the point tracking measurements in the moving pendulum experiment, the derived modulus matching closely to the datasheet from the tensile test, and the close agreement of the human arm flexion results underscore the effectiveness of the color tracking algorithm for analyzing a variety of experimental measurements. The high precision, reliability, and low-cost of this technique make it a valuable tool for engineers in materials testing applications and displacement tracking. The findings demonstrate that the color tracking algorithm exhibited higher precision and low noise, indicating its potential to provide more accurate and reliable measurements compared to comparative point tracking methods. Without wishing to be bound to any particular theory of operation, this enhanced performance may be attributed to the algorithm’s ability to locate the centroid of the specified color, allowing for precise tracking and analysis of material deformation.

[0062] In terms of measurement resolution, the color tracking algorithm offers subpixel displacement accuracy. This means that the algorithm can detect and measure displacements with a resolution finer than a single pixel. Subpixel displacement resolution typically translates into measurements on the order of 200 pm.

[0063] The use of the color tracking algorithm as set forth herein provides several advantages over the comparative examples. One advantage is in affordability and availability. The primary equipment required for this technique is a color camera, and in the study described above,a mobile phone camera (Google Pixel 2 XL) was used. In video mode, this is only 2 megapixels in camera resolution; therefore, the experiments above demonstrate that even relatively low resolution solutions work. The cost of the camera used in the experiments was a refurbished phone for a total cost of $109, which is significantly lower than specialized cameras employed in comparative optical measurement methods such as DIC. In addition, many people own or have easy access to a mobile phone with a camera. By leveraging the capabilities of a mobile phone camera, the color tracking algorithm eliminates the need for this specialized equipment and opens up new possibilities for its application in various fields where cost constraints may have previously limited the adoption of accurate measurement techniques. Additionally, the low-cost nature of the approach makes it particularly attractive as an educational tool, allowing students to gain hands- on experience with optical techniques without significant financial barriers. Furthermore, the methodology set forth herein may operate with only information regarding the fiducial markers, and without information regarding the object under inspection.

[0064] Beyond displacement and strain measurement, the color tracking algorithm also is applicable for measuring motion in other contexts. As demonstrated by the example of measuring the movement of a human arm, the algorithm can be applied to capture the motion of complex objects. This versatility opens up possibilities for its adoption in biomechanics research, rehabilitation studies, sports related tracking, and other fields where precise motion tracking is essential.

[0065] The performance of the color tracking algorithm can be affected by variations in lighting conditions. Changes in illumination, shadows, or reflections may introduce noise and affect the accuracy of the measurements. Careful control of lighting conditions may be performed to ensure reliable and consistent results. However, the use of the fiducial marker (as opposed to alternative tracking modalities, such as feature detection) provides some resilience to lighting condition variations, in addition to resilience to stress and strain.

[0066] While the color tracking algorithm offers subpixel displacement resolution, the spatial resolution may be limited by the camera’s capabilities and image quality. Thus, while the above description shows the efficacy of the algorithm even with comparatively low-resolution cameras, higher-resolution cameras may be used to capture more detailed deformation patterns, especially in materials with small-scale features or fine spatial variations.

[0067] The above experiments illustrate the applicability and effectiveness of color tracking algorithms applied to two-dimensional images acquired by a single image sensor. However, in other implementations the color tracking algorithms and methodologies set forth herein may be applied to three-dimensional images acquired by either a single or multiple image sensors. In one particular example, two smartphones may be provided according to a known physical relationship (e.g., a set distance apart) in order to capture stereoscopic images, which may be used to provide stereoscopic analysis of the target object.

[0068] Moreover, the field of view of the camera used in the study, such as a mobile phone camera, may be restricted. If the fiducial marker displaces out of the field of view, measurements are no longer able to be taken. This limitation may be overcome by employing stitching techniques to combine multiple images and extend the coverage area.

[0069] Example Implementations

[0070] FIG. 7 is a block diagram illustrating an example of a machine upon which one or more aspects of embodiments of the present invention can be implemented.

[0071] Referring to FIG. 7, an aspect of an embodiment of the present invention includes, but not limited thereto, a system, method, and computer readable medium that provides a) accurate displacement and strain measurements using a cost-effective color camera and color tracking algorithm and / or b) accurate displacement and strain measurement using a color tracking algorithm. FIG. 7 illustrates a block diagram of an example machine 400 upon which one or more embodiments (e.g., discussed methodologies) can be implemented (e.g., run).

[0072] Examples of machine 400 can include logic, one or more components, circuits (e g., modules), or mechanisms. Circuits are tangible entities configured to perform certain operations. In an example, circuits can be arranged (e.g., internally or with respect to external entities such as other circuits) in a specified manner. In an example, one or more computer systems (e.g., a standalone, client or server computer system) or one or more hardware processors (processors) can be configured by software (e.g., instructions, an application portion, or an application) as a circuit that operates to perform certain operations as described herein. In an example, the software can reside (1) on a non-transitory machine readable medium or (2) in a transmission signal. In an example, the software, when executed by the underlying hardware of the circuit, causes the circuit to perform the certain operations.

[0073] In an example, a circuit can be implemented mechanically or electronically. For example, a circuit can comprise dedicated circuitry or logic that is specifically configured to perform one or more techniques such as discussed above, such as including a special-purpose processor, an FPGA, or an ASIC. In an example, a circuit can comprise programmable logic (e.g., circuitry, as encompassed within a general-purpose processor or other programmable processor) that can be temporarily configured (e.g., by software) to perform the certain operations. It will be appreciated that the decision to implement a circuit mechanically (e.g., in dedicated and permanently configured circuitry), or in temporarily configured circuitry (e.g., configured by software) can be driven by cost and time considerations.

[0074] Accordingly, the term “circuit” is understood to encompass a tangible entity, be that an entity that is physically constructed, permanently configured (e g., hardwired), or temporarily (e.g., transitorily) configured (e.g., programmed) to operate in a specified manner or to perform specified operations. In an example, given a plurality of temporarily configured circuits, each of the circuits need not be configured or instantiated at any one instance in time. For example, where the circuits comprise a general -purpose processor configured via software, the general- purpose processor can be configured as respective different circuits at different times. Software can accordingly configure a processor, for example, to constitute a particular circuit at one instance of time and to constitute a different circuit at a different instance of time.

[0075] In an example, circuits can provide information to, and receive information from, other circuits. In this example, the circuits can be regarded as being communicatively coupled to one or more other circuits. Where multiple of such circuits exist contemporaneously, communications can be achieved through signal transmission (e.g., over appropriate circuits and buses) that connect the circuits. In embodiments in which multiple circuits are configured or instantiated at different times, communications between such circuits can be achieved, for example, through the storage and retrieval of information in memory structures to which the multiple circuits have access. For example, one circuit can perform an operation and store the output of that operation in a memory device to which it is communicatively coupled. A further circuit can then, at a later time, access the memory device to retrieve and process the stored output. In an example, circuits can be configured to initiate or receive communications with input or output devices and can operate on a resource (e.g., a collection of information).

[0076] The various operations of method examples described herein can be performed, at least partially, by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform the relevant operations. Whether temporarily or permanently configured, such processors can constitute processor-implemented circuits that operate to perform one or more operations or functions. In an example, the circuits referred to herein can comprise processor-implemented circuits.

[0077] Similarly, the methods described herein can be at least partially processor- implemented. For example, at least some of the operations of a method can be performed by one or processors or processor-implemented circuits. The performance of certain of the operations can be distributed among the one or more processors, not only residing within a single machine, but deployed across a number of machines. In an example, the processor or processors can be located in a single location (e.g., within a home environment, an office environment or as a server farm), while in other examples the processors can be distributed across a number of locations.

[0078] The one or more processors can also operate to support performance of the relevant operations in a "cloud computing" environment or as a "software as a service” (SaaS). For example, at least some of the operations can be performed by a group of computers (as examples of machines including processors), with these operations being accessible via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., Application Program Interfaces (APIs).)

[0079] Example embodiments (e.g., apparatus, systems, or methods) can be implemented in digital electronic circuitry, in computer hardware, in firmware, in software, or in any combination thereof. Example embodiments can be implemented using a computer program product (e.g., a computer program, tangibly embodied in an information carrier or in a machine readable medium, for execution by, or to control the operation of, data processing apparatus such as a programmable processor, a computer, or multiple computers).

[0080] A computer program can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a software module, subroutine, or other unit suitable for use in a computing environment. A computer program can be deployed to be executed on one computer or on multiple computers at one site or distributed across multiple sites and interconnected by a communication network.

[0081] In an example, operations can be performed by one or more programmable processors executing a computer program to perform functions by operating on input data and generating output. Examples of method operations can also be performed by, and example apparatus can be implemented as, special purpose logic circuitry (e.g., an FPGA or an ASIC).

[0082] The computing system can include clients and servers. A client and server are generally remote from each other and generally interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. In embodiments deploying a programmable computing system, it will be appreciated that both hardware and software architectures require consideration. Specifically, it will be appreciated that the choice of whether to implement certain functionality in permanently configured hardware (e.g., an ASIC), in temporarily configured hardware (e.g., a combination of software and a programmable processor), or a combination of permanently and temporarily configured hardware can be a design choice. Below are set out hardware (e.g., machine 400) and software architectures that can be deployed in example embodiments.

[0083] In an example, the machine 400 can operate as a standalone device or the machine 400 can be connected (e.g., networked) to other machines. In a networked deployment, the machine 400 can operate in the capacity of either a server or a client machine in server-client network environments. In an example, machine 400 can act as a peer machine in peer-to-peer (or other distributed) network environments. The machine 400 can be a personal computer (PC), a tablet PC, a set-top box (STB), a Personal Digital Assistant (PDA), a mobile telephone, a web appliance, a network router, switch or bridge, or any machine capable of executing instructions (sequential or otherwise) specifying actions to be taken (e.g., performed) by the machine 400. Further, while only a single machine 400 is illustrated, the term “machine” shall also be taken to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein.

[0084] Example machine (e.g., computer system) 400 can include a processor 402 (e.g., a CPU, a GPU, or both), a main memory 404, and a static memory 406, some or all of which can communicate with each other via a bus 408. The machine 400 can further include a display unit 410, an alphanumeric input device 412 (e.g., a keyboard), and a user interface (UI) navigation device 411 (e.g., a mouse). In an example, the display unit 810, input device 417 and UI navigationdevice 414 can be a touch screen display. The machine 400 can additionally include a storage device (e.g., drive unit) 416, a signal generation device 418 (e.g., a speaker), a network interface device 420, and one or more sensors 421, such as a global positioning system (GPS) sensor, compass, accelerometer, or other sensor.

[0085] The storage device 416 can include a machine readable medium 422 on which is stored one or more sets of data structures or instructions 424 (e.g., software) embodying or utilized by any one or more of the methodologies or functions described herein, and may be any type of “memory” described above. The instructions 424 can also reside, completely or at least partially, within the main memory 404, within static memory 406, or within the processor 402 during execution thereof by the machine 400. In an example, one or any combination of the processor 402, the main memory 404, the static memory 406, or the storage device 416 can constitute machine readable media.

[0086] While the machine readable medium 422 is illustrated as a single medium, the term “machine readable medium” can include a single medium or multiple media (e.g., a centralized or distributed database, and / or associated caches and servers) that configured to store the one or more instructions 424. The term “machine readable medium” can also be taken to include any tangible medium that is capable of storing, encoding, or carrying instructions for execution by the machine and that cause the machine to perform any one or more of the methodologies of the present disclosure or that is capable of storing, encoding or carrying data structures utilized by or associated with such instructions. The term “machine readable medium” can accordingly be taken to include, but not be limited to, solid-state memories, and optical and magnetic media. Specific examples of machine readable media can include non-volatile memory, including, by way of example, semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices); magnetic disks such as internal hard disks and removable disks; magneto-optical disks; and CD- ROM and DVD-ROM disks.

[0087] The instructions 424 can further be transmitted or received over a communications network 426 using a transmission medium via the network interface device 420 utilizing any one of a number of transfer protocols (e.g., frame relay, IP, TCP, UDP, HTTP, etc.). Example communication networks can include a local area network (LAN), a wide area network (WAN), a packet data network (e.g., the Internet), mobile telephone networks (e.g., cellular networks), Plain Old Telephone (POTS) networks, and wireless data networks (e.g., IEEE 802.11 standards familyknown as Wi-Fi®, IEEE 802.16 standards family known as WiMax®), peer-to-peer (P2P) networks, among others. The term “transmission medium” shall be taken to include any intangible medium that is capable of storing, encoding or carrying instructions for execution by the machine, and includes digital or analog communications signals or other intangible medium to facilitate communication of such software.

[0088] FIG. 7 shows a flow chart of an example optical inspection method 500. The method 500 includes a loop of operations that are repeated for multiple image frames (e.g., for a multiple frames of a video) of a target object captured by an image sensor, the target object including at least one fiducial marker. In one example, the method 500 may be performed by the machine 400 of FIG. 6. In the loop, at operation 510 an image frame is received. The image frame includes an n-dimensional array of pixel values corresponding to a plurality of colors. For example, the image frame may include a three-dimensional array of red pixel values, blue pixel values, and green pixel values. At operation 520, the image frame is converted to a color-adjusted image, the color-adjusted image including a two-dimensional array of pixel values corresponding to a first color of the plurality of colors. In one example, the first color is green and the conversion may be accomplished by the process set forth above, including applying Equation (1). In general, the first color may be any one of the plurality of colors, and may be the same color as the fiducial marker. Operation 520 may include applying at least one of the removal factor a or the threshold value T, as described above. At operation 530, at least one centroid is determined corresponding to at least one region in the color-adjusted image. The loop including operations 510-530 may repeat any number of times, each iteration corresponding to a different image frame.

[0089] Using information from the loop, at operation 540 the centroid determined at operation 530 is tracked across multiple iterations of the loop (i.e., across two or more image frames). Based on parameters of the centroid, at operation 550 at least one mechanical parameter is output. The mechanical parameter may be at least one of a displacement, a strain, or a pose of the target object. The particular mechanical parameter output may be related to the number of fiducial markers, centroids, and / or regions. For example, one fiducial may be used to determine and track a centroid for a single region; in this case, the mechanical parameter may be a displacement (or velocity, acceleration, etc.) of the target object. In another example, two fiducials may be used to determine and track centroids for two regions; in this case, the mechanical parameter may be a relative displacement or strain of the target object. In yet another example,three or more fiducials may be used to determine and track centroids for three or more regions; in this case, the mechanical parameter may be a pose of the target object. The method 500 may output mechanical parameters corresponding to a lower number of fiducials, centroids, and / or regions than the total number present in the system. For example, if the target object includes three fiducials, the method 500 may output the strain between two fiducials (or multiple strains between different pairs of fiducials) in addition to or instead of the pose.

[0090] The method 500 may be performed in a real-time or post-processing manner. In the real-time (or near real-time) manner, operations 540 and 550 may be performed immediately after the second iteration of the loop of operations 510-530, and may be continually performed or updated after successive iterations. This may provide real time (or, to account for the time required to perform computational operations, near real time) information regarding the mechanical parameter. In the post-processing manner, operations 540 and 550 may be performed at later time, once all iterations of the loop of operations 510-530 have been performed. In either case, the device performing the processing operations 510-550 may be the same as or different from the device capturing the images (i.e., may be the smartphone having the camera, or a different device). Moreover, in either case, different devices may be configured to perform different ones or sets of the operations of the method 500. For example, some pre-processing operations (e.g., operation 520) may be performed by a first device, and subsequent operations may be performed by a second device.

[0091] As set forth above, the present disclosure sets forth a cost-effective technique for accurate displacement and strain measurement using a color tracking algorithm. By leveraging the capabilities of a mobile phone camera and a green fiducial marker, the algorithm provides a low- cost alternative to comparative optical measurement methods. Other types of cameras may be implemented other than mobile cameras.

[0092] The findings of the experiments set forth above demonstrate the effectiveness of the color tracking algorithm in capturing displacement and strain with higher precision and low noise. The simplicity and affordability of the color tracking algorithm make it an attractive solution for displacement and strain measurement in various fields. The use of a mobile phone camera significantly reduces financial barriers and enhances accessibility. The simplicity and ease of implementation makes this technique a valuable educational tool for students learning about optical measurement techniques. The potential applications of the color tracking algorithm extendbeyond material science, as demonstrated by the example of measuring human arm movement. Its versatility in capturing motion opens up possibilities for its utilization in biomechanics, rehabilitation, and related fields.

[0093] In summary, while the present invention has been described with respect to specific embodiments, many modifications, variations, alterations, substitutions, and equivalents will be apparent to those skilled in the art. The present invention is not to be limited in scope by the specific embodiment described herein. Indeed, various modifications of the present invention, in addition to those described herein, will be apparent to those of skill in the art from the foregoing description and accompanying drawings. Accordingly, the invention is to be considered as limited only by the spirit and scope of the disclosure (and claims) including all modifications and equivalents.

[0094] Still other embodiments will become readily apparent to those skilled in this art from reading the above-recited detailed description and drawings of certain exemplary embodiments. It should be understood that numerous variations, modifications, and additional embodiments are possible, and accordingly, all such variations, modifications, and embodiments are to be regarded as being within the spirit and scope of this application. For example, regardless of the content of any portion (e.g., title, field, background, summary, abstract, drawing figure, etc.) of this application, unless clearly specified to the contrary, there is no requirement for the inclusion in any claim herein or of any application claiming priority hereto of any particular described or illustrated activity or element, any particular sequence of such activities, or any particular interrelationship of such elements. Moreover, any activity can be repeated, any activity can be performed by multiple entities, and / or any element can be duplicated. Further, any activity or element can be excluded, the sequence of activities can vary, and / or the interrelationship of elements can vary. Unless clearly specified to the contrary, there is no requirement for any particular described or illustrated activity or element, any particular sequence or such activities, any particular size, speed, material, dimension or frequency, or any particular interrelationship of such elements. Accordingly, the descriptions and drawings are to be regarded as illustrative in nature, and not as restrictive. Moreover, when any number or range is described herein, unless clearly stated otherwise, that number or range is approximate. When any range is described herein, unless clearly stated otherwise, that range includes all values therein and all sub ranges therein. Any information in any material (e.g., a United States / foreign patent, United States / foreign patent application, book, article, etc.) that has been incorporated by reference herein, is only incorporatedby reference to the extent that no conflict exists between such information and the other statements and drawings set forth herein. In the event of such conflict, including a conflict that would render invalid any claim herein or seeking priority hereto, then any such conflicting information in such incorporated by reference material is specifically not incorporated by reference herein.

Claims

CLAIMSWhat is claimed is:

1. An optical inspection method, comprising: for a plurality of image frames of a target object captured by an image sensor, the target object including at least one fiducial marker: receiving a respective image frame, the image frame including an n-dimensional array of pixel values corresponding to a plurality of colors, converting the image frame to a color-adjusted image, the color-adjusted image including a two-dimensional array of pixel values corresponding to a first color of the plurality of colors, and determining at least one centroid corresponding to at least one region in the color- adjusted image; tracking the at least one centroid across at least two of the plurality of image frames; and outputting an indication of a mechanical parameter of the target object based on the tracking.

2. The method of claim 1, wherein the operation of converting includes applying at least one of a removal factor and a threshold value to pixel values of the image frame corresponding to a second color and a third color of the plurality of colors.

3. The method of claim 1, wherein the at least one fiducial marker is one fiducial marker, the at least one centroid corresponding to the at least one region is one centroid corresponding to one region, and the mechanical parameter is a displacement of the target object.

4. The method of claim 1, wherein the at least one fiducial marker is two fiducial markers,the at least one centroid corresponding to the at least one region is two centroids corresponding to two regions, and the mechanical parameter is a displacement or a strain of the target object.

5. The method of claim 1, wherein the at least one fiducial marker is three or more fiducial markers, the at least one centroid corresponding to the at least one region is three or more centroids corresponding to three or more regions, and the mechanical parameter is a pose of the target object.

6. The method of claim 1, wherein the first color corresponds to a color of the fiducial marker.

7. The method of claim 1, wherein the plurality of colors are red, blue, and green, and wherein the first color is green.

8. The method of claim 1, wherein the image sensor is a smartphone camera.

9. An optical inspection system, comprising: an imaging system configured to capture a plurality of image frames of a target object, the target object including at least one fiducial marker; and at least one processor configured to: for the plurality of image frames: receive a respective image frame, the image frame including an n- dimensional array of pixel values corresponding to a plurality of colors, convert the image frame to a color-adjusted image, the color-adjusted image including a two-dimensional array of pixel values corresponding to a first color of the plurality of colors, and determine at least one centroid corresponding to at least one region in the color-adjusted image,track the at least one centroid across at least two of the plurality of image frames, and output an indication of a mechanical parameter of the target object based on the tracking.

10. The system of claim 9, wherein, to convert the image frame, the at least one processor is configured to apply at least one of a removal factor and a threshold value to pixel values of the image frame corresponding to a second color and a third color of the plurality of colors.

11. The system of claim 9, wherein the at least one fiducial marker is one fiducial marker, the at least one centroid corresponding to the at least one region is one centroid corresponding to one region, and the mechanical parameter is a displacement of the target object.

12. The system of claim 9, wherein the at least one fiducial marker is two fiducial markers, the at least one centroid corresponding to the at least one region is two centroids corresponding to two regions, and the mechanical parameter is a displacement or a strain of the target object.

13. The system of claim 9, wherein the plurality of colors are red, blue, and green, and wherein the first color is green.

14. The system of claim 9, wherein the imaging system is a camera of a smartphone, and the at least one processor is a processor of the smartphone.

15. The system of claim 9, wherein the at least one processor is configured to receive the respective image frame, convert the image frame to the color-adjusted image, and determine the at least one centroid in real time or near real time.

16. A non-transitory computer-readable medium storing instructions that, when executed by at least one processor of a computer, cause the computer to perform operations comprising: for a plurality of image frames of a target object captured by an image sensor, the target object including at least one fiducial marker: receiving a respective image frame, the image frame including an n-dimensional array of pixel values corresponding to a plurality of colors, converting the image frame to a color-adjusted image, the color-adjusted image including a two-dimensional array of pixel values corresponding to a first color of the plurality of colors, and determining at least one centroid corresponding to at least one region in the color- adjusted image; tracking the at least one centroid across at least two of the plurality of image frames; and outputting an indication of a mechanical parameter of the target object based on the tracking.

17. The non-transitory computer-readable medium of claim 16, wherein the operation of converting includes applying at least one of a removal factor and a threshold value to pixel values of the image frame corresponding to a second color and a third color of the plurality of colors.

18. The non-transitory computer-readable medium of claim 16, wherein the at least one fiducial marker is one fiducial marker, the at least one centroid corresponding to the at least one region is one centroid corresponding to one region, and the mechanical parameter is a displacement of the target object.

19. The non-transitory computer-readable medium of claim 16, wherein the at least one fiducial marker is two fiducial markers,the at least one centroid corresponding to the at least one region is two centroids corresponding to two regions, and the mechanical parameter is a displacement or a strain of the target object.

20. The non-transitory computer-readable medium of claim 16, wherein the at least one fiducial marker is three or more fiducial markers, the at least one centroid corresponding to the at least one region is three or more centroids corresponding to three or more regions, and the mechanical parameter is a pose of the target object.

Images